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ABSTRACT 


-  \ 

The  Best  Replacement  Factor  (BRF)  is  used  primarily  in  Coordinated  Ship¬ 
board  Allowance  List  (COSAL)  and  load  list  requirements  determination.  Re¬ 
cently,  many  inventory  problems  including  stock  shortages,  stock  excesses,  and 
churn  have  been  blamed  on  the  BRF.  To  identify  what  problems  are  associated 
with  the  current  method  used  to  compute  the  BRF,  we  took  a  multi-faceted  ap¬ 
proach.  First,  we  discussed  the  two  forecasting  methods  available  under  Re- 
systemization  and  then  analyzed  the  volatility  and  trend  of  the  current  BRF 
Casualty  Report  (CASREP)  items  and  problem  equipments  were  included  in  the 
analysis.  We  then  examined  alternative  methods  to  forecast  the  BRF  and  com¬ 
pared  them  with  the  current  methods.  The  best  alternative  was  selected  based 
on  its  performance  in  terms  of  stability,  accuracy  with  respect  to  bias  and 
extremes,  and  the  magnitude  of  errors.  Performance  was  evaluated  separatelv 
for  items  identified  as  readiness  contributors.  Test  COSALs  were  built  with 
the  current  BRFs  and  the  proposed  BRFs  to  evaluate  the  impact  of  the  change. 

We  recommend  using  the  exponential  probability  distribution  to  adjust  the 
Technical  Replacement  Factor  (TRF)  during  the  development  period,  and  after 


the  development  period,  exponential  smoothing  with  a  data  windsorizing  system 

to  handle  extreme  values.  . / _ _  —  - - — 
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EXECUTIVE  SUMMARY 


1.  Background.  The  Best  Replacement  Factor  (BRF)  is  used  primarily  in  Coor¬ 
dinated  Shipboard  Allowance  List  (COSAL)  and  load  list  requirements  determina¬ 
tion.  Recently,  many  inventory  problems  including  stock  shortages,  stock  ex¬ 
cesses,  and  churn  have  been  blamed  on  the  BRF. 

2.  Objective .  To  identify  and  investigate  problems  associated  with  the  cur¬ 
rent  method  used  to  compute  the  BRF.  After  these  problems  have  been  identi¬ 
fied  and  investigated,  develop  alternatives  to  improve  the  BRF  forecast. 

3.  Approach .  The  first  phase  began  with  an  analytical  discussion  of  Exponen¬ 
tial  Smoothing  and  the  Ratio  Method  (the  two  methods  available  under  Resys- 
temization) .  Next,  we  measured  current  BRF  volatility  over  time  by  examining 
BRF  changes  and  trends.  Casualty  Report  (CASREP)  requisitions  were  then  ana¬ 
lyzed.  From  the  analysis  of  CASREP  items,  we  identified  and  examined  items 
that  appeared  to  have  major  support  problems.  Finally  we  examined  a  single 
weapon  system  (PHALANX)  in  an  attempt  to  identify  specific  BRF  problems. 

Taking  the  information  gained  in  the  first  phase,  we  developed  forecast 
methods  to  correct/improve  the  BRF.  These  new  methods,  as  well  as  the  current 
methods,  were  ranked  based  on  their  performance  with  respect  to  all  items  and 
readiness  contributors.  Performance  was  related  to  stability,  accuracy  with 
respect  to  bias  and  extremes,  and  magnitude  of  the  errors.  To  evaluate  the 
impact  of  any  change,  we  compared  the  best  method  to  the  Benchmark  (current 
forecast  method)  COSAL  computed  with  Modified  Fleet  Logistics  Support  Improve¬ 
ment  Program  (MOD-FLSIP)  rules. 

4.  Findings .  BRF  instability  and  trend  are  primarily  related  to  the  subjec¬ 
tive  portion  of  the  forecast  (Technical  Replacement  Forecasts  (TRFs) ,  ordnance 
freeze,  manual  changes  and  overrides).  Eliminating  the  TRF,  ordnance  freeze, 


manual  changes  and  overrides  from  the  forecast  produced  more  stable  forecasts 
that  trended  closer  to  the  actual  Navy  Maintenance  and  Material  Management 
(3M)  average  usage  rates  computed  from  all  available  data. 

Using  the  exponential  probability  distribution  to  adjust  the  TRF  during 
the  development  period,  and  exponential  smoothing,  with  a  data  windsorizing 
system  to  handle  extreme  values  after  the  development  period,  provided  the 
best  forecast  with  respect  to  stability,  bias  and  extremes,  and  the  magnitude 
of  the  errors.  These  results  were  consistent  for  all  items  and  the  subset  of 
items  classified  as  the  readiness  contributors. 

The  proposed  method  appears  to  slightly  reduce  COSAL  range  and  effective¬ 
ness.  However,  we  believe  the  improved  stability  and  accuracy  will  provide 
long  term  benefits  in  reduced  COSAL  churn  which  tends  to  create  long  supply 
and/or  outfitting  deficiencies. 

5.  Recommendation .  We  recommend  using  the  exponential  probability  distribu¬ 
tion  to  adjust  the  TRF  during  the  development  period.  After  the  development 
period,  use  exponential  smoothing  with  a  data  windsorizing  system  to  handle 
extreme  values . 
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I .  INTRODUCTION 


The  Best  Replacement  Factor  (BRF)  is  used  primarily  in  Coordinated  Ship¬ 
board  Allowance  List  (COSAL)  and  load  list  requirements  determination.  Re¬ 
cently,  many  inventory  problems,  including  stock  shortages,  stock  excesses, 
and  churn  have  been  blamed  on  the  BRF.  Some  concerns  are  unrelated  to  the 
method  used  to  compute  the  BRF.  Poor  reporting  by  activities,  bad  configura¬ 
tion  data,  and  manual  changes  can  lead  to  distorted  forecasts.  Other  concerns 
are  related  directly  to  the  current  method  used  to  compute  the  BRF.  It  has 
been  suggested  that  using  different  data  sources  when  Navy  Maintenance  and 
Material  Management  (3M)  data  is  unavailable  and  using  the  current  forecast 
method  (Exponential  Smoothing  with  a  weight  of  .4)  which  reacts  quickly  to 
changes,  create  an  unstable  forecast. 

This  concern  has  grown  since  Readiness  Based  Sparing  (RBS)  models  were 
approved  for  use  in  conjunction  with  the  COSAL  Modified  Fleet  Logistics  Sup¬ 
port  Improvement  Program  (MOD-FLSIP)  model.  Under  MOD-FLSIP  rules,  an  item's 
requirements  are  based  only  on  its  expected  demand  (BRF  times  the  item's  acti¬ 
vity  population).  Changes  in  an  item's  BRF  only  affect  that  item's  require¬ 
ments.  With  an  RBS  model,  requirements  are  determined  based  on  the  relative 
demands  and  prices  of  all  items  being  optimized;  i.e.,  items  compete  against 
each  other  based  on  their  expected  demand  and  cost.  A  change  in  one  item's 
BRF  can  affect  not  only  its  own  requirements  but  those  of  other  items  as  well 
As  a  result,  the  Ratio  Method  was  proposed  as  a  more  stable  alternative  for 
Resystemization .  The  Ra^io  Method  is  the  ratio  of  the  cumulative  3M  usage  to 
the  cumulative  3M  population.  This  method  is  not  yet  approved  for  use,  nor 
has  it  been  shown  to  provide  more  cost-effective  results. 

Therefore,  our  initial  phas«_  of  this  study  was  to  determine  if  any  of 
these  concerns  are  valid.  After  identifying  and  analyzing  problems  associated 


with  the  present  BRF,  we  then  propose  and  evaluate  methods  to  correct/improve 
the  forecast. 

II.  APPROACH 

In  this  section,  we  describe  the  method,  data  and  performance  measurements 
used  in  our  analysis 
A.  METHOD . 

We  begin  the  first  phase  with  an  analytical  comparison  of  Exponential 
Smoothing  and  the  Ratio  Method,  the  two  methods  available  under  Resystemiza- 
tion.  Next,  we  measure  BRF  volatility  over  time  by  examining  BRF  changes  and 
trends.  CASREP  items  are  analyzed  to  identify  and  examine  problem  items.  Fi¬ 
nally,  we  analyze  items  on  a  sample  problem  equipment  (PHALANX)  in  an  attempt 
to  identify  specific  BRF  problems.  Throughout  the  data  analysis,  we  include 
an  analytical  discussion  comparing  what  we  observed  with  the  theory. 

Using  the  information  gained  in  the  first  phase,  forecast  methods  are  de¬ 
veloped  to  improve  the  BRF.  These  methods  are  ranked  based  on  their  perform¬ 
ance  with  respect  to  all  items  and  only  those  items  identified  as  readiness 
contributors.  Performance  for  items  on  two  selected  equipments  is  also  ana¬ 
lyzed.  To  measure  the  impact  of  any  changes,  the  best  method  is  compared  to 
the  benchmark  COSAL  (computed  with  the  current  forecast  method  and  with  MOD- 
FLSIP  rules)  for  several  test  ships. 

B.  DATA. 

The  Naval  Sea  Logistics  Center  (NSLC)  provided  us  with  their  10  year 
BRF  history  file  (1978-1987)  and  Casualty  Report  (CASREP)  requisitions  for  the 
period  1  January  1986  to  31  December  1987.  The  10  year  history  file  contains 
the  BRF  and  item  demand  and  average  population  for  3M,  Mobile  Logistics  Sup¬ 
port  Force  (MLSF) ,  and  system  data.  We  obtained  PHALANX  data  from  CACI . 
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These  data  were  used  to  analyze  potential  BRF  problems.  The  HR!'  history  file 
was  also  used  to  compute  BRFs  using  alternative  procedures. 

To  measure  the  impact  of  any  BRF  change,  Navy  Ships  Parts  Control  Center 
(SPCC)  provided  us  with  candidate  files  from  the  July  1988  COSAL  extract  for 
the  following  ships: 

•  CG  24  -  USS  REEVES 

•  DDG  6  -  USS  BARNEY 

•  DDG  16  -  USS  STRAUSS 

•  DDG  993  -  USS  SPRUANCE 

•  FFG  13  -  USS  SPRAGUE 

•  LPD  14  -  USS  TRENTON 

NSLC  provided  us  with  3M  demand  data  from  1  August  1988  to  10  November  1988  to 
evaluate  the  test  COSALs .  NSLC  also  provided  Weapon  System  File  (WSF)  ex¬ 
tracts  for  two  equipments;  the  PHALANX  (Allowance  Parts  List  (APL)  number 
006090052)  and  an  engine  (APL  number  L665360264).  These  data  were  used  to 
more  closely  evaluate  BRFs  on  sample  equipments.  To  determine  the  impact  on 
readiness,  NSLC  provided  a  list  of  Federal  Supply  Classifications  (FSCs)  to 
define  which  items  they  considered  potential  readiness  contributors. 

C.  PERFORMANCE  MEASURES. 

Model  range  and  requisition  effectiveness,  in  conjunction  with  cost, 
was  used  to  measure  the  impact  on  the  COSAL.  Model  range  effectiveness  is  the 
percent  of  COSAL  candidate  items  with  demand  that  was  stocked  by  M0D-FLS1P. 
Model  requisition  effectiveness  is  the  percent  of  requisitions  for  COSAL  can¬ 
didate  items  that  was  satisfied  using  the  COSAL  quantity. 

BRF  forecast  performance  was  ranked  based  on  stability,  accuracy  with  res¬ 
pect  to  bias  and  extremes,  and  accuracy  with  respect  to  the  magnitude  of  the 
forecast  errors.  The  criteria  used  to  rate  the  forecast  methods  is  varied  and 
at  times  contradictory.  That  is,  the  forecast  method  selected  may  or  mav  not 
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be  the  most  stable  or  accurate,  but  it  will  provide  the  best  combination  of 
stability  and  accuracy  given  the  alternative  methods.  It  is  important  to  note 
that  all  the  measures  look  in  hindsight  to  see  how  the  yearly  BRFs  compare  to 
the  average  3M  rate  over  time.  The  average  BRF  is  based  on  5-10  years  data 
depending  on  the  history  available  for  the  item. 

The  following  paragraphs  explain  the  forecast  performance  measures  in  de¬ 
tail  . 

1.  Stability .  We  measured  stability  in  terms  of  the  number  of  times 
(years)  the  forecast  is  within  95%  confidence  limits  of  the  average  item  3M 
usage  rate  (computed  from  the  10  year  history  file).  The  more  stable  a  fore¬ 
casting  method  is,  the  more  often  its  forecast  will  be  within  these  limits. 

0L  '  *  +  '  (  yi"  ) 

LL  - 5  -  *  ( jr ) 

where 

UL  -  the  upper  confidence  limit 

LL  -  the  lower  confidence  limit 

1  N 

X  -  the  average  usage  rate  for  the  nth  item  computed  as  -  Z  3M  usage 

N  i-1 

ratej 

N  =  the  number  of  observations  for  the  nth  item 

S  -  the  standard  deviation  for  the  nfch  item  computed  as 
1  N 

-  Z  (3M  usage  ratei  -  X)2 

i-i 

t  -  t-statistic  used  to  define  the  95%  confidence  limits 
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2.  Accuracy  with  Resnect  to  Bias  and  Extremes.  We  measured  accuracy 
with  respect  to  bias  and  extremes  by  the  Mean  Error  (ME) .  The  ME  measures  how 
forecasts  err.  A  negative  ME  indicates  a  forecast  tends  to  overforecast , 
while  a  positive  ME  denotes  a  tendency  to  underf orecast .  When  an  overforecast 
occurs,  the  item  has  a  better  opportunity  to  be  stocked  (or  stocked  with  a 
larger  quantity)  and  there  is  less  risk  of  stockout  (the  part  is  likely  to  be 
available);  however,  investment  may  be  larger  than  required  (excess  material 
may  be  bought).  With  an  underforecast ,  the  opposite  is  true  (i.e.,  the  item 
will  have  a  worse  opportunity  to  be  stocked,  there  is  a  higher  risk  of  stock¬ 
out,  but  less  excess  material  will  be  bought).  Forecast  performance  will  be 
ranked  by  the  size  of  the  over forecast .  Small  overforecasts  are  ranked  high¬ 
est.  Large  underforecasts  are  ranked  lowest. 

1  N 

HE  =  -  £  (3H  usage  rate*  -  F*) 

1=1 

where 

ME  =  the  mean  error  for  the  nth  item 
N  -  the  number  of  observations  for  the  nu>  item 
F1  -  the  ith  forecast  for  the  nth  item 

3M  usage  ratej  =  the  actual  3M  experienced  rate  for  the  nth  item  during 
the  ith  observation 

3.  Accuracy  with  Respect  to  Magnitude  of  the  Errors.  We  measure  ac¬ 
curacy  with  respect  to  ..he  magnitude  of  the  forecast  errors  by  the  Mean  Square 
Error  (MSE) .  Here  we  are  interested  in  how  close  the  forecast  is  to  the  3M 
usage  rate.  Consequently,  forecast  performance  is  ranked  on  the  ability  to 
forecast  precisely  (with  small  errors). 
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where 


MSE 


1 

-  S  (3M  usage  rateA  -  Fj)2 
N  i=l 


MSE  -  the  mean  square  error  for  the  nth  item 
N  -  the  number  of  observations  for  the  nth  item 
Fj  -  the  ith  forecast  for  the  nth  item 

3M  usage  ratei  -  the  actual  3M  experienced  rate  for  the  nth  item  during 
the  i'h  observation 


III.  FINDINGS 

This  section  is  comprised  of  two  parts.  In  the  first  segment,  we  will 
identify  and  analyze  problems  associated  with  the  current  BRF.  In  the  next 
section,  we  will  propose  alternatives  for  improving  the  BRF. 

A.  BRF  PROBLEM  ANALYSIS. 

To  determine  what  problems  exist,  and  if  the  concerns  mentioned  pre¬ 
viously  are  valid,  we  took  a  multi-faceted  approach.  First,  we  examined  the 
two  forecasting  methods  available  under  Resystemization;  i.e.,  Exponential 
Smoothing  and  the  Ratio  Method.  Next,  we  analyzed  the  volatility  and  trend  of 
the  current  BRF.  Then,  we  examined  CASREP  requisition  data.  Using  the  CASREP 
data,  we  identified  and  investigated  problem  items.  Finally,  we  analyzed 
items  on  a  problem  equipment  (the  PHALANX  data  set) . 

1.  Exponential  Smoothing  Versus  the  Ratio  Method.  The  current  BRF 
and  the  Ratio  Method  use  different  techniques  to  transition  from  the  TRF  to  an 
experience-based  demand  rate.  They  can  also  use  different  data  sources.  For 
complete  details  see  APPENDIX  B.  Our  discussion  focuses  on  the  techniques 
used  to  phase  observations  into  the  forecast  with  an  emphasis  on  the  degree  of 
adaptability  and  stability  of  the  methods.  The  tradeoff  between  adaptability 
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and  stability  is  usually  controlled  by  the  amount  of  influence  or  weight  pla¬ 
ced  on  recent  observations  versus  older  observations.  APPENDIX  C  provides  a 
detailed  explanation  of  how  these  weights  are  developed  for  each  method. 

Exponential  Smoothing  is  a  moving  average  technique.  The  name  is  derived 
from  how  the  influence  or  weight  placed  on  an  observation  changes  over  time; 
i.e.,  it  declines  exponentially.  For  example,  viewing  TABLE  I  we  see  that  tie 
weight  of  the  first  observation  (viewed  for  the  first  time  in  year  two)  decli¬ 
nes  from  .4  to  .007  over  a  nine  year  period  using  a  smoothing  weight  of  .4, 
and  declined  from  .1  to  .043  using  a  smoothing  weight  of  .1. 

The  degree  of  adaptability  and  stability  for  this  forecast  method  is  de¬ 

termined  by  the  smoothing  weight  (a).  With  a  higher  weight,  more  emphasis  is 
placed  on  the  more  recent  observation  than  on  the  older  observations;  there¬ 
fore,  the  forecast  adapts  quicker  to  changes  than  with  a  lower  weight,  but  is 
less  stable.  A  lower  weight  places  more  emphasis  on  older  observations  than 
the  more  recent  observations.  Consequently,  the  forecast  does  not  react  as 
quickly  as  with  a  higher  weight  (less  adaptable),  but  is  more  stable.  This 
can  be  seen  by  viewing  TABLE  I.  TABLE  1  provides  a  comparison  of  the  emphasis 
placed  on  observations  for  up  to  a  10  year  period,  for  high  (.4)  and  low  (.1) 
smoothing  weights.  For  example,  in  the  10th  year,  64%  (.24  +  .4)  of  the  high 
smoothing  weight  forecast  is  from  the  two  most  recent  observations  compared  to 
19%  (.09  +  .1)  for  the  low  smoothing  weight  forecast.  The  oldest  two  values 

(TRF  and  first  observation)  account  for  only  1.7%  (.01  +  .007)  of  the  high 

smoothing  weight  forecast  compared  to  41%  (.367  +  .043)  of  the  low  smoothing 
weight  forecast.  Thus,  the  lower  smoothing  weight  takes  longer  to  "washout" 
the  effects  on  the  initial  forecast  (TRF) . 
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TABLE  I 


EXPONENTIAL  SMOOTHING  WEIGHTS 
WEIGHT  =  .4 


YEAR 

TRF 

OBS 

1 

OBS 

2 

OBS 

3 

OBS 

4 

OBS 

5 

OBS 

6 

OBS 

7 

OBS 

8 

OBS 

9 

Y1 

Y2 

0.400 

Y3 

0.360 

0.240 

Y4 

0.216 

0.144 

0.240 

0.400 

Y5 

0.130 

0.086 

0.144 

0.240 

0.400 

Y6 

0.078 

0.052 

0.086 

0.144 

0.240 

0.400 

Y7 

0.047 

0.031 

0.052 

0.086 

0.144 

0.240 

0.400 

Y8 

0.028 

0.019 

0.031 

0.052 

0.086 

0.144 

0.240 

Y9 

0.017 

0.011 

0.019 

0.031 

0.052 

0.086 

0.144 

0.240 

0.400 

Y10 

0.010 

0.007 

0.011 

0.019 

0.031 

0.052 

0.086 

0.144 

0.240 

0.400 

WEIGHT  =  .1 


YEAR 

TRF 

OBS 

1 

OBS 

2 

OBS 

3 

OBS 

4 

OBS 

5 

OBS 

6 

OBS 

7 

OBS 

8 

OBS 

9  ' 

Y1 

Y2 

nfi 

Y3 

0.810 

0.090 

Y4 

0.729 

0.090 

0.100 

Y5 

0.656 

0.073 

0.081 

0.090 

0.100 

Y6 

0.590 

0.066 

0.073 

0.081 

0.090 

Y7 

0.551 

0.059 

0.066 

0.073 

0.081 

0.090 

0.100 

Y8 

0.478 

0.053 

0.059 

0.066 

0.073 

0.081 

0.090 

Y9 

0.430 

0.048 

0.053 

0.059 

0.066 

0.073 

0.081 

0.090 

0.100 

Y10 

0.367 

0.043 

0.048 

0.053 

0.059 

0.066 

0.073 

0.081 

0.090 

0.100 

The  relationship  between  the  size  of  the  smoothing  weight  and  stability 
becomes  murky  if  distortion  occurs;  i.e.,  we  observe  unusual  values.  A  dis¬ 
torted  observation  will  have  a  greater  initial  impact  on  the  forecast  with  a 
larger  weight  than  with  a  lower  weight,  but  the  distortion  will  also  be  fil¬ 
tered  out  more  rapidly  with  the  higher  weight  than  the  lower  weight.  For  ex¬ 
ample  (viewing  TABLE  I),  an  observation  viewed  for  the  first  time  in  year  two 
accounts  for  40%  of  the  higher  weight  forecast  as  compared  to  10%  for  the 
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lower  weight  forecast.  However,  after  four  years  (year  six),  that  same  obser¬ 
vation  will  now  account  for  more  of  the  low  weight  forecast  than  the  high 
weight  forecast  (6.6%  versus  5.2%).  By  the  10th  year,  that  observation  will 
have  almost  no  influence  on  the  high  weight  forecast  (.7%)  as  compared  to  4.3% 
for  the  low  weight  forecast. 

Exponential  Smoothing  is  a  widely  used  technique  for  many  reasons.  First, 
it  has  been  shown  to  provide  viable  results  as  compared  to  other  methods  in 
numerous  studies.  Second,  it  is  relatively  easy  to  understand  and  use.  Fin¬ 
ally,  it  only  requires  two  data  points  to  compute  a  forecast;  i.e.,  the  old 

forecast  and  the  most  recent  observation.  Consequently,  Exponential  Smooth¬ 
ing  is  the  method  most  widely  used  for  large  inventory  systems. 

The  main  problems  associated  with  Exponential  Smoothing  are  developing  a 
starting  point  and  determining  the  smoothing  weight.  Smoothing  weights  are 
usually  chosen  analytically  based  on  the  desired  tradeoff  between  stability 
and  adaptability.  The  starting  point  is  usually  the  first  observation  or  an 
average  of  initial  observations  over  time.  As  can  be  seen  by  TABLE  1,  the 
current  policy  (Exponential  Smoothing  with  a  weight  of  .4  and  the  TRF  as  a 
starting  point)  places  a  large  influence  on  the  TRF.  The  TRF  always  has  more 
influence  on  the  BRF  than  the  first  usage  rate  observed  and  almost  as  much 
weight  as  the  second  observed  usage  rate.  It  takes  five  years  of  data  to 
reduce  the  TRF's  impact  on  the  forecast  to  less  than  10%.  Lowering  the 

smoothing  weight  places  an  even  larger  influence  on  the  TRF.  After  10  years, 

the  TRF  still  accounts  for  37%  of  the  BRF.  More  weight  is  given  the  TRF  than 
the  four  most  recent  observations  combined  (six  through  nine) .  Since  TRFs  are 
only  a  best  guess,  this  may  cause  a  problem  with  the  current  BRF. 

The  Ratio  Method  was  developed  by  the  NSLC  as  an  alternative  to  the  cur¬ 
rent  method.  With  this  method,  the  observations  are  not  smoothed  into  the 
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TRF.  The  TRF  is  the  forecast  (the  BRF)  until  either  two  demands  have  or 
should  have  occurred.  Then,  the  TRF  is  discarded,  and  the  Ratio  Method  fore¬ 
cast  becomes  the  BRF.  The  Ratio  Method  divides  total  accumulated  demand  by 
total  population.  This  method  claimed  to  weigh  each  observation  equally  and 
thus  improve  BRF  stability.  However,  as  shown  in  APPENDIX  C,  this  is  not 
necessarily  the  case.  The  weight  of  a  yearly  observation  is  dependent  on 
changes  in  the  population  size  used  to  compute  the  usage  rate.  If  population 
is  constant,  then  all  observations  are  weighted  equally.  If  population  var¬ 
ies,  the  weight  of  each  observation  is  determined  by  the  size  of  the  popula¬ 
tion  generating  that  observation.  For  analytical  purposes,  we  doubled  the 
population  size  each  year  and  halved  the  population  size  each  year,  for  up  to 
a  10  year  period  (see  TABLE  II).  When  population  size  increases,  the  forecast 
becomes  more  adaptable  and  less  stable;  and,  when  population  size  decreases, 
the  forecast  becomes  less  responsive  and  more  stable.  Population  sizes  used 
to  calculate  usage  rates  change  yearly.  They  increase  and  decrease  from  year 
to  year.  Therefore,  the  degree  of  adaptability  and  stability  changes  from 
year  to  year,  is  beyond  the  control  of  the  user,  and  is  unknown. 
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TABLE  II 

RATIO  METHOD  WEIGHTS  ASSUMING  CONSTANT  POPULATIONS 


1.000 

0.500 

0.333 

0.250 

0.200 

0.167 

0.143 

0.125 

0.111 

0.100 


0.500 

0.333 

0.250 

0.200 

0.167 

0.143 

0.125 

0.111 

0.100 


.333 

.250 

0.200 

0.167 

0.143 

0.125 

0.111 

0.100 


0.250 

0.200 

0.167 

0.143 

0.125 

0.111 

0.100 


0.200 

0.167 

0.143 

0.125 

0.111 

0.100 


67 

43  0.143 

.25  0.125  0.125 

0.111  0.111  0.111 

0.100  0.100  0. 


RATIO  METHOD  WEIGHTS  ASSUMING  POPULATIONS  DOUBLE  EACH  YEAR 


0.067 

0.032 

0.016 

0.008 

0.004 

0.002 

0.001 


0.065 

0.032 

0.016 

0.008 

0.004 

0.002 


.571 

.267 

.129 

.063 

0.031 

0.016 

0.008 

0.004 


.533 
.258 
0.127 
0.063 
0.031 
0.016 
0.008 


0.516 

0.254 

0.126 

0.063 

0.031 

0.016 


.508 
.252 
0.125 
0.063 
0.031 


0.502 

0.250 

0.125 


RATIO  METHOD  WEIGHTS  ASSUMING  POPULATIONS  HALVED  EACH  YEAR 


0.508 

0.504 

0.502 

0.501 

0.500 


0.333 

0.286 

0.267 

0.258 

0.254 

0.252 

0.251 

0.250 

0.250 


43 
33 
29 
0,127 
0.126 
0.125 
0.125 
0.125 


0.067 

0.065 

0.063 

0.063 

0.063 

0.063 

0.063 


0.032 

0.032 

0.031 

0.031 

0.031 

0.031 


6 
6 

0.016 

0.016 

0.016 


0.008 

0.008 

0.008 

0.008 


0.002 

0.002  0.001 


In  summary,  with  Exponential  Smoothing,  the  user  controls  the  degree  of 
stability  and  adaptability  of  the  forecast  and  the  influence  of  the  TRF 
through  the  smoothing  weight.  A  large  weight  produces  less  stable  more  res¬ 
ponsive  forecasts.  Lowering  the  weight  increases  stability  and  decreases  res¬ 
ponsiveness.  Distorted  values  have  a  greater  initial  impact  on  the  forecast 
with  a  higher  weight  than  a  lower  weight,  but  they  are  also  filtered  out  of 
the  forecast  quicker  with  a  higher  weight  than  a  lower  weight.  With  the  Ratio 
Method,  the  degree  of  stability  and  adaptability  depends  on  changes  in  popula¬ 
tion  size;  therefore,  it  is  beyond  the  control  of  the  user  and  is  unknown. 
Population  increases  create  more  responsive,  less  stable  forecasts.  Popula¬ 
tion  decreases  produce  more  stable,  less  responsive  forecasts. 

2.  BRF  Volatility  and  Trend.  We  defined  an  item's  BRF  as  volatile 
if  it  had  a  percentage  change  of  greater  than  25%  between  any  two  consecutive 
years.  Only  one  change  between  any  two  consecutive  years  was  necessary  to  be 
defined  as  volatile.  We  used  Pearson's  correlation  coefficients  (r)  (see 
APPENDIX  D  for  r  computation)  to  measure  trend  over  the  entire  10  year  BRF 
history  file  (1,269,864  items,  including  some  obsolete  items). 

Based  on  our  criteria,  69%  of  the  items  had  volatile  BRFs.  We  found  that 
most  of  the  changes  were  decreases.  The  largest  group  (13%  of  the  volatile 
items)  had  only  one  decrease  and  eight  years  of  constant  BRFs.  The  next  lar¬ 
gest  groups  (6%,  4%,  and  3%  of  the  volatile  items)  had  nine  straight  years  of 
BRF  decreases,  eight  years  of  decreases  and  one  BRF  increase,  and  eight  years 
of  decreases  and  one  year  of  no  change,  respectively. 

Items  with  stable  BRFs  had  little  or  no  history.  Of  the  items  with  stable 
BRFs,  43%  had  no  BRFs  for  the  first  nine  years;  i.e.,  they  had  no  chance  to  be 
volatile.  We  used  statistical  t-tests  and  F  tests  (see  APPENDIX  D)  to  deter¬ 
mine  that  the  volatile  items  had  significantly  larger  average  3M  populations 
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Chan  the  stable  items.  As  shown  in  TABLE  111,  the  3M  populations  vary  more 
and  are  larger  for  the  volatile  items  than  the  stable  items.  This  implies 
that  the  volatile  items  have  more  opportunity  for  demand  history. 


TABLE  III 
3M  POPULATION 


STABLE  ITEMS 

VOLATILE  ITEMS 

AVERAGE 

STD  DEVIATION 

140.0 

8,973.4 

409.2 

13,936.3 

In  the  previous  section,  we  found  that  the  current  policy  places  a  large 
influence  on  the  TRF.  Therefore,  we  recomputed  the  BRF  using  the  data  on  the 
history  file  with  Exponential  Smoothing  but  without  TRFs ,  manual  changes, 
ordnance  freezes,  and  overrides;  i.e.,  the  subjective  portion  of  the  forecast 
was  eliminated.  The  smoothing  weight  was  held  to  .4  (current  policy)  and  the 
first  observed  usage  rate  was  the  starting  point.  (NOTE:  Prior  to  1985,  if  a 
usage  rate  was  greater  than  the  BRF,  a  weight  of  .8  was  used.  Since  this  po¬ 
licy  has  been  changed,  and  most  of  the  volatile  items  had  decreases  rather 
than  increases,  we  chose  to  use  the  current  policy  (a  =  .4)  over  the  entire 
time  period.)  TABLE  IV  shows  that  eliminating  the  subjective  portion  of  the 
forecast  produced  about  2.5  times  more  items  with  stable  BRFs .  This  proves 
that  the  TRFs,  manual  changes,  ordnance  freezes,  and  overrides  cause  most  of 
the  BRF  unstability,  not  Exponential  Smoothing. 
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TABLE  IV 


BRF  VOLATILITY 


RECOMPUTED 

SYSTEM 

BRF 

BRF 

(W/0  SUBJECTIVES) 

VOLATILE 

878,246 

310,667 

STABLE 

391,618 

959,197 

We  computed  Pearson's  correlation  coefficients  (r)  to  measure  trend  over 
time  for  the  System  BRFs ,  3M  usage  rates  (3M  demand  divided  by  3M  population), 
and  Recomputed  BRFs  without  subjectives.  We  used  this  statistic  to  determine 
whether  the  actual  3M  rates  are  trending,  and  whether  the  current  System  BRF 
and/or  the  Recomputed  BRFs  show  the  same  trend  pattern. 

The  correlation  coefficients,  rounded  to  the  nearest  tenth,  for  the  System 
BRF,  the  3M  usage  rate,  and  Recomputed  BRF  without  subjectives  are  displayed 
in  TABLE  V.  We  also  provide  the  number  and  percent  of  items  associated  with 
each  value  of  r.  A  value  of  0  indicates  no  trend.  Viewing  TABLE  V,  we  see 
that  System  BRFs  (current  method  as  found  on  the  10  year  history  file)  are 
more  likely  to  have  a  trend  than  3M  usage  rates.  The  correlation  coefficients 
were  zero  for  only  7%  of  the  System  BRFs  as  compared  to  78%  for  3M  usage 
rates.  For  the  TABLE  V  items,  Pearson's  coefficient  tells  us  we  can  be  over 
95%  confident  that  items  with  an  r  greater  than  .5  or  less  than  -.5  are  trend¬ 
ing.  Thus  we  are  over  95%  confident  that  57%  of  the  item  System  BRFs  have 
trend  (33%  negative  trend  and  24%  positive  trend)  compared  to  4%  of  the  item 
3M  usage  rates  (2%  negative  trend  and  2%  positive  trend).  Eliminating  the 
subjective  portion  of  the  forecast  yields  forecasts  which  have  correlation  co¬ 
efficients  closer  to  3M  usage  rates;  that  is,  the  Recomputed  BRFs  trend  closer 
to  actual  3M  usage  rates  than  the  System  BRFs.  Now,  73%  of  the  item  forecasts 


have  values  of  zero  for  r.  Only  15%  of  the  item  forecasts  have  a  trend  (5% 
negative  trend  and  10%  positive  trend) . 


SYSTEM  BRF 


NUMBER  PERCENT 


13,073 

169,307 

137,383 

61,708 

36,811 

97,761 

18,984 

31,583 

23,122 

23,354 

90,958 

23,036 

25,851 

44,540 

31,376 

132,490 

29,052 

75,444 

113,550 

88,291 

180 


3M  USAGE  RATE 

NUMBER 

PERCENT 

32 

0 

926 

0 

3,313 

0 

8,246 

1 

9,516 

1 

22,027 

2 

20,604 

2 

20,222 

2 

17,874 

1 

19,571 

2 

990,562 

78 

22,436 

2 

21,464 

2 

23,529 

2 

25,295 

2 

34,115 

12,531 

10,819 

3 

1 

1 

5,215 

0 

1,516 

0 

41 

0 

RECOMPUTED  BRFs 
W/0  SUBJECTIVES 


NUMBER  PERCENT 


2,838 
29,762 
12,804 
8 , 189 
16,102 
10,926 
7,823 
14,720 
7,499 
12,676 
923,459 
8,771 
16,210 
10,554 
19,819 
40,319 
41,659 
44,332 
27,374 
12,780 
1,238 


In  summary,  examining  the  volatility  and  trend  of  System  BRFs,  we  found 
that  items  with  volatile  BRFs  tend  to  have  a  small  number  of  changes  (which 
are  decreasing)  and  items  with  stable  BRFs  have  little  to  no  history.  The 
System  BRF  trends  more  than  the  actual  3M  usage  rate;  i.e.,  57%  of  the  item 
System  BRFs  show  trend  versus  4%  for  item  3M  usage  rates.  Recomputing  BRFs 
from  the  data  without  subjectives  (i.e.,  eliminating  TRFs ,  manual  changes, 
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ordnance  freezes,  and  overrides)  produces  a  more  stable  forecast  and  one  that 
trends  closer  to  actual  3M  usage  rates.  The  number  of  items  with  stable  BRFs 
increased  by  about  150%  and  the  items  with  trend  decreased  from  57%  to  15%. 

This  proves  that  volatility  and  trend  are  largely  due  to  TRFs,  manual  changes, 
ordnance  freezes,  and  overrides,  not  to  Exponential  Smoothing  or  alternative 
data  sources. 

3.  CASREP  Reauisition/Problem  Item  Analysis.  We  reviewed  sample  CAS- 
REP  "problem"  items  to  determine  whether  the  BRF  was  a  major  cause  of  the  CAS¬ 
REP.  Most  CASREP  requisitions  only  occurred  once  in  the  two  year  period.  Of 
the  43,024  items  with  CASREP  requisitions,  71%  only  occurred  once.  Based  on 
the  criteria  of  the  previous  section,  59%  of  the  CASREP  items  had  volatile 
System  BRFs  (64%  of  these  had  only  one  CASREP  requisition) .  Out  of  the  57 
ship  types  reporting  CASREP  requisitions,  54%  of  the  CASREPs  were  from  five 
ship  types;  i.e.,  SSN,  DD,  DDG ,  FF,  and  FFG. 

Using  Pearson's  correlation  coefficients  from  the  previous  section  (r  > 

| . 5 | ) ,  we  investigated  whether  or  not  there  was  a  problem  with  System  BRFs  and 
3M  usage  rates  trending  in  opposite  directions.  This  was  not  a  problem.  Of 
the  1,001  items  with  BRFs  having  positive  trends  and  3M  usage  rates  with  nega¬ 
tive  trends,  12%  had  CASREPs  (57%  of  these  had  only  one  CASREP  requisition). 
The  number  of  CASREP  requisitions  ranged  from  zero  to  17.  We  found  5,180 
items  having  System  BRFs  with  negative  trends  and  3M  usage  rates  with  positive 
trends.  Only  14%  of  these  items  had  CASREP  requisitions  (62%  of  these  occur¬ 
red  only  once).  The  number  of  these  requisitions  ranged  from  zero  to  36.  For 
most  of  these  5,180  items,  the  TRF  is  larger  than  the  3M  usage  rate,  and  over 
time  the  System  BRF  and  3M  usage  rates  are  trending  toward  each  other. 

For  analysis  purposes,  we  defined  a  problem  item  as  an  item  with  more  than 
50  CASREPs  in  the  two  year  period.  Based  on  our  criteria,  26  items  were  iden- 
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tified  as  problem  items.  We  readily  concede  that  items  with  50  or  less  CAS- 
REPs  may  also  be  problem  items;  however,  limiting  the  sample  to  a  small  number 
of  items  allowed  us  to  do  more  in-depth  analysis. 

Some  of  the  problems  we  found  with  these  items  were  not  related  to  the 
BRF.  First,  3M  population  data  was  not  always  available  for  these  items  on 
the  10  year  history  file,  even  during  the  period  when  the  CASREPs  occurred. 
Second,  these  items'  BRFs  tended  to  be  large.  The  largest,  smallest,  most  re¬ 
cent,  and  oldest  BRFs  for  the  NIINs  of  these  items  are  displayed  in  TaBLE  VI, 
along  with  the  number  of  years  of  BRF  history  (an  asterisk  denotes  items  which 
had  gaps  where  the  history  was  missing  or  unavailable) .  For  62%  of  these 
items,  the  smallest  BRF  was  at  least  .1,  and  for  69%  of  the  problem  items,  the 
most  recent  BRF  was  at  least  .1.  Under  MOD-FLSIP  rules,  an  item  with  an  ex¬ 
pected  demand  (BRF  *  Population)  of  at  least  .1  and  an  Item  Mission  Essen¬ 
tiality  Code  (IMEC)  greater  than  2  has  at  least  one  Minimum  Replacement  Unit 
(MRU)  stocked.  Since  these  items  had  more  than  50  CASREPs,  we  can  assume  that 
their  IMECs  were  3  or  more.  Therefore,  even  with  a  population  as  low  as  one, 
we  can  assume  at  least  one  MRU  was  stocked  for  most  of  these  items.  This  im¬ 
plies  that  the  problems  associated  with  these  items  may  be  related  to  the 
population  data,  or  the  model  (not  enough  depth),  or  supply  availability  (as¬ 
sets  not  available  in  the  system).  Finally,  all  of  the  problem  items  are  re- 
pairables.  Repairable  items  (even  appearing  multiple  times  in  an  equipment) 
must  be  CASREPed  separately.  Therefore,  one  equipment  failure  can  generate 
multiple  CASREPs  for  the  same  item,  as  well  as  different  items. 
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TABLE  VI 


PROBLEM  ITEMS'  BRFs 


NUN 

MIN  BRF 

001221493 

0.350 

003635619 

0.260 

004662258 

1.000 

006137235 

0.016 

008690109 

0.003 

009495591 

0.180 

010224765 

0.020 

010498099 

0.230 

010543301 

0.110 

010673688 

0.100 

010804324 

0.120 

010822927 

0.066 

010898895 

0.045 

010965933 

0.500 

011041308 

0.240 

011099480 

2.000 

011187279 

0.038 

011442593 

0.100 

011513058 

0.170 

011575880 

0.001 

011577009 

0.046 

011586889 

0.100 

011637619 

0.034 

011769727 

0.001 

011979852 

0.497 

012126298 

0.309 

0.270 

0.490 

0.780 

0.164 

0.650 

0.170 

0.330 

0.497 

0.309 


^Denotes  history  not  consecutive 


0.380 

0.970 

1.300 

0.020 

0.003 

0.300 

0.038 

0.230 

0.110 

0.160 

0.120 

0.160 

0.093 

0.670 

0.300 

2.100 

0.110 

0.250 

0.490 

0.001 

0.062 

0.100 

0.034 

0.001 

0.497 

0.309 


OLDEST 


0.350 

0.260 

1.000 

0.110 

0.005 

0.190 

0.020 

0.600 

0.500 

0.100 

0.800 

0.100 

0.300 

0.500 

0.431 

2.000 

0.120 

0.100 

0.170 

0.780 

0.046 

0.650 

0.170 

0.330 

0.497 

0.309 


NUMBER  OF 
YEARS  BRF  HIST 


We  found  inconsistent  usage  rates  for  these  items,  both  over  time  and  be¬ 
tween  3M,  MLSF,  and  system  data.  The  autocorrelation  coefficients  (this  is 
another  way  to  measure  trend  which  requires  a  visual  confirmation)  for  all 
three  data  source  usage  rates  dropped  to  zero  after  one  time  lag.  This  indi¬ 
cates  the  data  is  stationary  (a  flat  nonsloping  trend).  Therefore,  either  the 
Exponential  Smoothing  method  or  the  Ratio  Method  is  appropriate,  as  neither  of 
these  methods  allow  for  a  linear  trend. 
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Most  of  the  items  have  significant  BRF  histories  (see  TAB1.K  Vi). 


For  sunk' 


items,  the  BRF  history  is  not  consistent.  However,  62%  of  the  problem  items 
have  nine  or  more  consecutive  years  of  history.  For  these  items,  the  TRF 
should  account  for  no  more  than  1.7%  of  the  forecast  (see  TABLE  I).  There¬ 
fore,  the  TRF  should  not  be  a  factor  in  these  BRFs. 

Although  not  shown  in  this  report,  we  plotted  the  System  BRF,  the  Recom¬ 
puted  BRF,  the  Ratio  Method  BRF,  and  the  usage  rates  over  time.  The  system 
forecast  was  the  actual  BRF  value  on  the  10  year  history  file.  In  most  in¬ 
stances,  the  System  BRF  exceeded  the  usage  rates.  The  System  BRF  did  not  al¬ 
ways  follow  an  item's  usage  rate  pattern.  For  some  items,  a  BRF  change  was 
exaggerated  as  compared  to  the  actual  usage  rate  change.  This  indicates  man¬ 
ual  manipulation  of  BRFs  for  some  reason. 

We  next  analyzed  whether  or  not  the  Ratio  Method  provided  a  more  stable 
forecast  than  Exponential  Smoothing.  A  previous  study  (reference  1  to  APPEN¬ 
DIX  A)  compared  the  current  method  and  the  Ratio  Method  with  respect  to  accur¬ 
acy  and  COSAL  effectiveness.  However,  it  did  not  distinguish  differences  in 
the  forecasts  due  to  TRF  transitioning.  We  wanted  to  compare  method  to  method 
with  no  TRF  transitioning  effect.  The  Recomputed  BRF  without  subjectives 
allows  us  to  compare  pure  Exponential  Smoothing  with  the  Ratio  Method.  The 
Ratio  Method  BRF  was  computed  using  3M  data  only  (as  per  APPENDIX  B) ,  but  no 
TRF  transition  was  made.  Therefore,  if  an  item  had  3M  data  only,  the  Recom¬ 
puted  BRF  and  the  Ratio  Method  BRF  would  use  the  same  data. 

Looking  at  the  forecasts  over  time,  both  the  Ratio  Method  and  Exponential 
Smoothing  BRFs  varied,  but  not  more  than  the  actual  usage  rates.  To  determine 
if  one  forecast  varied  more  than  the  other,  we  conducted  an  F  test  on  their 
variances  for  each  item  (see  APPENDIX  D) .  The  F  test  indicated  that  the  Ex¬ 
ponential  Smoothing  BRF  varied  more  than  the  Ratio  BRF  (at  the  5%  level  of 
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significance)  for  five  of  che  26  items.  For  two  items,  this  was  a  matter  of 
using  alternative  data  versus  3M  data  only  forecasts.  One  of  these  two  items 
never  had  any  3M  data  (other  sources  were  available);  i.e.,  there  was  no  ratio 
forecast  (in  reality  it  would  be  the  TRF) .  The  other  item  had  3M  data  for 
only  the  first  two  years  (1978  and  1979);  therefore,  the  forecast  was  not  up¬ 
dated  by  the  Ratio  Method,  but  was  with  Exponential  Smoothing.  The  other 
three  items  contained  extreme  observations  that  should  have  qualified  for  a 
manual  review.  Ratio  Method  did  not  vary  more  than  the  Exponential  Smoothing 
for  any  of  the  26  items. 

In  summary,  we  found  CASREPed  requisitions  occurred  for  a  relatively  small 
number  of  items  (43,024)  and  most  occurred  only  once  in  the  two  year  history 
(71%).  Items  with  BRF  and  3M  usage  rates  with  opposite  trends  were  not  a  fac¬ 
tor.  Twenty-six  items  were  identified  as  problem  items  for  in-depth  analysis. 
Population  data  (3M)  was  not  always  available  for  these  items  on  the  10  year 
history  file.  Most  of  these  items  had  BRFs  large  enough  to  ensure  stockage, 
regardless  of  population  size,  and  the  BRF  was  often  larger  than  the  observed 
usage  rates.  Since  most  of  these  items  have  significant  BRF  histories,  the 
TRF  should  have  been  filtered  out  of  the  forecast  and  was  not  considered  a 
problem.  Usage  rates  for  the  problem  items  were  found  to  be  inconsistent  and 
stationary.  Evidence  of  manual  BRF  changes  were  observed.  Exponential 
Smoothing  with  a  weight  of  .4  tends  to  not  vary  more  than  the  Ratio  Method. 

4.  The  PHALANX  Data.  Here  we  narrowed  the  scope  of  our  analysis  by 
targeting  an  equipment.  The  PHALANX  was  selected  because  it  is  a  high  profile 
problem  equipment  with  a  reasonable  number  of  items  (3,336).  We  were  able  to 
match  82%  of  the  items  with  the  10  year  history  file.  First,  we  examined  the 
CASREP  requisitions.  Then  we  compared  Exponential  Smoothing  and  the  Ratio 
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Method.  Keep  in  mind,  our  purpose  here  was  to  gain  insight,  not  to  determine 
the  best  method. 

The  number  of  items,  percent  of  items  with  no  CASREP  requisitions,  per¬ 
cent  of  items  with  one  CASREP  requisition,  and  the  maximum  number  of  CASREP 
requisitons  are  shown  in  TABLE  VII  for  the  overall  equipment  and  grouped  by 
Cog.  The  majority  of  items  did  not  have  chronic  problems.  Only  12%  had  a 
CASREP  requisition  in  the  two  year  period.  Multiple  CASREPs  occurred  for  only 
5%  of  the  items.  If  an  item  had  a  problem,  it  probably  was  a  7  Cog  (repair¬ 
able  item).  For  7  cog  items,  18%  had  more  than  one  CASREP  requisition  in  the 
two  year  period,  compared  to  4%  and  2%  for  1  and  9  Cog  items.  The  number  of 
CASREP  requisitions  for  an  item  also  ranged  higher  for  7  Cog  than  for  1  and  9 
Cog.  For  the  PHALANX,  7  Cog  items  represented  a  small  number  of  items  (12%)  , 
but  a  large  proportion  of  CASREP  requisitions  (29%)  and  multiple  CASREPs 
(43%). 


TABLE  VII 

PHALANX  CASREP  ANALYSIS 


NO.  ITEMS 

PERCENT 

NO  CASREPS 

PERCENT 
ONE  CASREP 

MAX  CASREPs 

OVERALL 

3,336 

88 

7 

73 

1  COG 

711 

88 

8 

12 

7  COG 

396 

71 

11 

73 

9  COG 

2,220 

92 

6 

16 

Since  the  forecast  performance  measurements  (stability,  ME,  and  MSE)  are 
only  useful  when  comparing  methods  (we  cannot  say  a  method  is  stable  or  ac¬ 
curate;  we  can  only  say  a  method  is  more  stable  or  more  accurate  than  another 
method),  we  recomputed  BRFs  from  the  data  using  Exponential  Smoothing  and  the 
Ratio  Method  (the  methods  available  under  Resystemization) .  Again,  our  in- 
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terest  is  in  comparing  methods.  Therefore,  we  ignored  the  System  BRF  (which 
has  manual  changes,  ordnance  freezes,  overrides,  and  TRFs)  and  TRF  transition¬ 
ing.  For  Exponential  Smoothing,  we  used  the  two  weights  from  our  first  dis¬ 
cussion  comparing  Exponential  Smoothing  and  the  Ratio  Method;  i.e.,  .h  and  .1. 
As  computed  previously,  the  first  observed  usage  rate  was  the  starting  point 
and  the  forecast  was  updated  even  when  3M  data  was  unavailable.  The  Ratio 
forecast  was  computed  as  in  the  previous  section  with  3M  data  only. 

Since  we  have  a  large  number  of  items,  we  are  concerned  about  the  general 
tendency  of  the  performance  measurements.  Therefore,  we  will  use  the  mean  and 
median  to  measure  central  tendency  and  the  standard  deviation  to  measure  the 
dispersion.  The  mean  is  the  arithmetic  average.  Each  observation  has  the 
same  weight.  Therefore,  it  is  sensitive  to  a  few  extreme  values.  The  median 
is  simply  the  midpoint.  Half  the  observations  are  larger  and  the  other  half 
are  smaller  than  the  median.  Extreme  values  have  no  influence  on  the  median. 
The  standard  deviation  measures  how  items  vary  around  the  mean.  If  all  the 
observations  are  close  to  the  mean,  then  the  standard  deviation  will  be  small; 
i.e.,  there  is  little  dispersion.  We  calculated  the  mean,  median,  and  stan¬ 
dard  deviations  for  the  ME  and  MSE  for  the  two  forecast  methods,  along  with 
the  smallest  and  largest  errors. 

We  first  analyzed  the  ME  and  MSE  for  the  entire  PHALANX  data  set  (TABLE 
VIII).  To  eliminate  the  effect  of  using  alternative  data  when  3M  data  is  un¬ 
available,  we  then  performed  the  same  analysis  for  the  subset  of  PHAIANX  items 
containing  10  years  of  3M  data  (TABLE  IX).  Finally,  to  determine  the  impact 
of  updating  with  alternative  data  sources,  we  again  performed  the  same  analy¬ 
sis  with  the  entire  PHALANX  data  set,  but  we  did  not  update  the  forecast  when 
3M  data  was  unavailable  (TABLE  X) . 
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Interpreting  TABLE  VIII,  we  see  that  in  terms  of  ME,  Exponential  Smoothing 
with  a  weight  of  .4  is  best,  Exponential  Smoothing  with  a  weight  of  .1  is 
worst,  and  the  Ratio  Method  is  in  between.  We  also  note  that  the  Ratio  Method 
tends  to  underforecast  (positive  mean  ME),  while  the  other  methods  tend  to 
overforecast  (negative  mean  ME) .  The  three  methods  are  close  in  terms  of  the 
MSE. 


TABLE  VIII 

PHALANX  DATA  SET  -  ALL  THE  DATA 
3,336  ITEMS 


EXPONENTIAL  SMOOTHING 

RATIO 

METHOD 

WEIGHT  =  .4 

WEIGHT  -  .1 

ME 

MSE 

ME 

MSE 

ME 

MSE 

MEAN 

-0.289 

22,956.4 

-0.306 

19,782.3 

0 . 3  11 

20,838.8 

STD  DEV 

13.238 

1,302,679 

40.366 

1,079,588 

17.056 

1,189,444 

MAXIMUM 

125.641 

75,230,859 

1439.87 

62,259,066 

935.627 

68,695,920 

MEDIAN 

0 

.000008 

0 

.000009 

0 

.000007 

MINIMUM 

-727.2 

0 

-1799.09 

0 

-298.83 

0 

Since  the  Ratio  Method  does  not  update  the  forecast  when  3M  data  is  un¬ 
available  and  Exponential  Smoothing  does,  we  still  do  not  have  a  true  compari¬ 
son  of  methods.  We  need  to  eliminate  the  effect  of  using  alternative  data 
when  3M  data  is  unavailable.  TABLE  IX  contains  only  PHALANX  items  with  10 
years  of  3M  data.  Now,  all  the  forecasts  use  the  same  data,  and  we  can  obtain 
a  true  comparison  of  methods.  Interpreting  TABLE  IX,  we  see  that  for  the  ME, 
the  Ratio  Method  is  the  best,  while  Exponential  Smoothing  with  a  weight  of  .1 
is  worst.  The  same  results  hold  true  for  the  MSE. 
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TABLE  IX 


PHALANX  DATA  SET  -  10  YEARS  OF  3M  POP 
883  ITEMS 


EXPONENTIAL  SMOOTHING 

RATIO 

METHOD 

WEIGHT  -  .4 

WEIGHT  -  .1 

ME 

MSE 

ME 

MSE 

ME 

MSE 

MEAN 

-0.9 

1408.6 

-2.25 

3971.4 

-0.337 

912.1 

STD  DEV 

24.5 

40,781 

60.6 

116,812 

10.2 

26,121 

MAXIMUM 

1.9 

1,211,629 

10.7 

3,471,071 

41.6 

775,917 

MEDIAN 

-0.00004 

0.00057 

-0.0002 

0.0008 

0 

0.0006 

MINIMUM 

-727.2 

0 

-1799.09 

0 

-298.83 

0 

Since  these  items  had  10  years  of  3M  data,  they  are  a  subset  of  the  entire 
PHALANX  data  set.  That  is,  nothing  different  was  done  between  TABLES  VIII  and 
IX.  We  just  isolated  those  items  with  10  years  of  3M  data.  These  items  may 
not  give  us  a  clear  picture  of  the  impact  of  updating/not  updating  with  alter¬ 
native  data  sources.  Therefore,  we  again  recomputed  the  Exponential  Smoothing 
with  a  weight  of  .4  for  the  entire  PHALANX  data  set  and  used  only  3M  data; 
i.e.,  we  did  not  update  the  forecast  when  3M  data  was  unavailable.  (We  only 
recomputed  the  .4  forecasts  because  we  wanted  a  concise  comparison  and  .4  is 
the  current  policy.) 

TABLE  X  contains  the  comparisons  of  Exponential  Smoothing  (weight  of  .4) 
using  alternative  data  (benchmark)  and  3M  data  only  (alternative).  Comparing 
the  two  Exponential  Smoothing  alternatives,  we  see  that  for  both  the  ME  and 
MSE,  the  method  using  alternative  data  sources  is  better  than  the  3M  data  only 
alternative.  Overall,  the  evidence  in  TABLE  X  indicates  that  accuracy  is  lost 
by  not  updating  the  forecast  when  3M  data  is  unavailable.  This  loss  of  accur¬ 
acy  results  in  larger  overforecasts . 
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TABLE  X 


PHALANX  DATA  SET  -  ALL  THE  DATA 
3.336  ITEMS 

EXPONENTIAL  SMOOTHING  WEIGHT  =  .4 


EXPONENTIAL  SMOOTHING 

BENCHMARK 

ALTERNATIVE 

ME 

MSE 

ME 

MSE 

MEAN 

-0.289 

22,956.4 

-0.458 

25,885.5 

STD  DEV 

13.238 

1,302,679 

18.0422 

1,473,485 

MAXIMUM 

125.641 

75,230,859 

25.14 

85,097,483 

MEDIAN 

0 

.000008 

0 

.000006 

MINIMUM 

-727.2 

0 

-745.314 

0 

NOTE:  BENCHMARK  -  IF  NO  3M  POP  EXISTS,  THEN  UPDATE  FORECAST 

USING  ALTERNATIVE  DATA  SOURCES 
ALTERNATIVE  -  IF  NO  3M  POP  EXISTS,  THEN  NO  FORECAST 
UPDATE  IS  MADE 

Now,  we  will  look  at  stability.  Here  we  examined  the  fast  moving  items; 
i.e.,  items  with  10  years  of  positive  3M  usage  rates.  Ue  selected  these  344 
fast  moving  items  for  several  reasons.  As  discussed  previously,  we  used  con¬ 
fidence  limits  on  3M  data;  therefore,  items  with  no  3M  data  would  be  eliminat¬ 
ed.  Since  the  methods  use  the  same  data,  we  obtain  a  better  comparison.  Fin¬ 
ally,  the  fast  moving  items  should  be  the  most  variable.  Exponential  Smooth¬ 
ing  with  a  weight  of  .1  was  the  least  likely  method  to  have  an  item's  forecast 
exceed  the  limits  (51%  of  the  forecasts  were  within  the  confidence  limits  all 
10  years);  while,  Exponential  Smoothing  with  a  weight  of  .4  was  the  most  like¬ 
ly  method  to  have  a  forecast  exceed  the  limits  (only  10%  of  the  forecasts  were 
within  the  confidence  limits  all  10  years) .  The  Ratio  Method  was  in  between 
(36%  of  the  forecasts  were  within  the  limits  all  10  years).  However,  once  an 
item's  forecast  exceeded  the  limits,  it  was  most  likely  to  stav  outside  using 
Exponential  Smoothing  with  a  weight  of  .1.  It  was  least  likely  to  stav  out- 
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side  using  Exponential  Smoothing  with  a  weight  of  .4.  Both  the  Ratio  Method 
and  Exponential  Smoothing  with  a  weight  of  .1  had  item  forecasts  exceed  the 
limits  all  10  years.  Exponential  Smoothing  with  a  .4  weight  had  no  item  fore¬ 
cast  exceed  the  limits  for  more  than  six  years.  Exponential  Smoothing  with  a 
weight  of  .4  had  a  standard  deviation  of  1.3.  With  a  weight  of  .1.  the  mea¬ 
sure  of  dispersion  increased  to  3.8.  The  Ratio  Method  was  in  between. 

In  summary,  with  Exponential  Smoothing,  lowering  the  weight  increases  the 
stability  but  reduces  the  adaptability  of  the  forecast.  Extreme  values  will 
have  less  impact  on  the  forecast  initially  with  a  lower  weight  than  with  a 
higher  weight;  but  over  time,  extreme  values  have  a  larger  influence  with  the 
lower  weight  than  the  higher  weight.  This  confirms  the  theoretical  analysis 
presented  earlier.  Viewing  TABLES  VIII  and  IX,  we  see  the  extreme  values  ( the 
minimum  and  maximum  ME)  are  larger  for  the  lower  weight  than  the  higher  weight.. 
We  have  also  seen  that  forecasts  are  more  stable  with  a  lower  weight,  but  once 
they  become  distorted  are  less  likely  to  self  correct.  With  the  higher  weight 
the  forecasts  are  more  likely  to  become  distorted,  but  are  also  quicker  to  re¬ 
adjust.  The  Ratio  Method  is  somewhere  in  between. 

With  respect  to  CASREPs ,  the  PHALANX  data  findings  were  consistent.  Re¬ 
latively  few  PHALANX  items  had  CASREPs,  and  for  those  that  did,  most  only  oc¬ 
curred  once  in  the  two  year  period.  Only  5%  of  the  PHALANX  items  had  more 
than  one  CASREP.  Furthermore,  if  an  item  has  a  problem  it  is  most  likely  a 
repairable  item.  Repairable  items  (7  Cog)  represent  a  small  percent  of  tbe 
PHALANX  items  (12%),  but  a  large  proportion  of  the  CASREP  requisitions  (29%  ot 
all  CASREP  requisitions  and  43%  of  multiple  CASREPs) . 

Comparing  forecast  methods.  Exponential  Smoothing  has  a  tendency  to  over¬ 
forecast  and  the  Ratio  Method  tends  to  underforecast  when  considering  all 
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items.  Accuracy  may  be  lost  by  not  updating  forecasts  when  3M  data  is  un¬ 
available  . 

5.  Conclusions  -  BRF  Problem  Analysis.  With  Exponential  Smoothing, 
forecasts  tend  to  err  on  the  high  side  (overforecast).  For  the  Ratio  Method, 
forecasts  tend  to  err  on  the  low  side  (underforecast).  Eliminating  the  sub¬ 
jective  portion  of  the  forecast  (TRF,  manual  changes,  ordnance  freezes,  and 
overrides)  and  using  a  constant  smoothing  weight  of  .4,  we  obtained  BRFs  that 
trend  closer  to  actual  3M  usage  rates  than  current  BRFs  and  are  more  stable. 
These  forecasts  vary  no  more  than  the  Ratio  Method  forecasts.  Lowering  the 
smoothing  weight  to  .1  decreases  the  magnitude  of  overall  errors,  but  the 
magnitude  of  the  extreme  errors  increased.  The  forecast  was  less  likely  to 
exceed  95%  confidence  limits  for  average  3M  usage  rates  than  with  a  weight  of 
.4  or  the  Ratio  Method.  However,  once  the  limits  were  exceeded,  the  forecasts 
were  more  apt  to  self  adjust/correct  with  the  higher  weight  (.4)  than  with  the 
Ratio  Method  or  the  lower  weight  (.1).  3M  data  forecasts  may  not  be  as  accu¬ 
rate  as  forecasts  which  also  use  other  data  when  3M  is  unavailable.  These 
errors  were  generally  larger  overforecasts . 

Item  support  problems  may  not  be  related  to  the  BRF  but  to  data,  depth 
(the  model),  and/or  supply  availability.  CASREP  requisitions  occurred  for  a 
relatively  small  number  of  items  and  most  (71%)  only  occurred  once  in  a  two 
year  period.  Only  5%  of  the  PHALANX  data  had  more  than  one  CASRKP  requisit  ion, 
Most  of  the  CASREP  items  analyzed  had  BRFs  large  enough  to  ensure  shipboard 
stockage,  and  the  BRF  was  often  larger  than  the  observed  usage  rate. 
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B.  BRF  IMPROVEMENT. 


From  the  previous  section  we  found: 

•  BRF  instability  and  trend  primarily  result  from  the  subjective 
portion  of  the  forecast  -  TRF,  manual  changes,  ordnance  freezes, 
and  overrides. 

•  Short  term  extreme  data  observations  also  produce  excessive 
variation. 

•  Based  on  the  number  of  items  with  zero  correlation  coefficients 
(and  the  autocorrelation  coefficients  for  the  problem  items),  most 
items  are  stationary.  Therefore,  single  Exponential  Smoothing  is 
an  appropriate  forecast  method. 

•  3M  data  is  not  always  available  to  update  the  BRF. 

Therefore,  we  need  to: 

•  Examine  alternatives  for  transitioning  the  TRF  into  the  forecast. 

•  Minimize  the  impact  of  short  term  extreme  data  observations. 

•  Determine  the  impact  of  updating/not  updating  the  forecast  when  V 
data  is  unavailable. 

1.  Alternative  Forecast  Methods.  The  following  discussion  gives  a 
brief  overview  of  the  forecasting  methods  considered  as  alternatives  to  the 
current  method.  For  a  detailed  description  and  the  mathematical  formulas  see 
APPENDIX  B. 

The  DDG  52  RBS  working  group  is  using  (and  proposing)  a  Bovesian  method. 
Bayesian  forecast  methods  use  Bayes'  theorem  to  update  a  subjective  forecast 
(TRF)  with  objective  data  (usage  rate).  The  weights  are  similar  to  the  Ratio 
Method  (see  APPENDIX  C) .  The  tradeoff  between  adaptability  and  stabilitv  do 
pends  on  changes  in  population  size;  i.e.,  increases  in  population  size  pro- 
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duce  a  more  adaptable  forecast,  and  decreases  produce  a  more  stable  one.  The 
amount  of  influence  given  the  subjective  portion  versus  the  objective  portion 
of  the  forecast  depends  on  TRF  size.  For  small  values,  the  TRF  has  more  in¬ 
fluence  than  the  objective  data.  With  large  values,  the  objective  data  has 
more  influence  than  the  TRF.  With  the  Bayes  method,  the  weight  is  an  unknown 
variable  beyond  the  control  of  the  user. 

We  compute  the  Bayes  forecast  two  ways.  One  forecast  uses  3M  data  only. 
The  other  uses  alternative  data  sources  to  update  the  forecast  when  3M  data  is 
unavailable.  The  priority  of  data  selection  is  the  same  as  under  current  po¬ 
licy;  i.e.,  3M,  then  MLSF,  then  system  demand. 

Since  the  Ratio  Method  is  included  in  Resystemization ,  we  also  reevaluate 
it.  This  method  is  computed  as  discussed  previously  and  described  in  APPENDIX 
B.  (Currently,  the  Ratio  Method  is  defined  to  only  use  3M  data.) 

We  next  develop  an  alternative  specifically  designed  to  address  the  pro¬ 
blems  identified  earlier  in  the  report.  As  shown  previously,  one  of  the  pro¬ 
blems  with  the  current  procedure  is  related  to  TRF  transitioning.  The  fore¬ 
cast  can  also  be  improved  by  eliminating  excessive  short  term  data  observa¬ 
tions.  If  a  lower  weight  is  used,  accuracy  and  stability  improve  for  some 
items.  For  other  items,  they  get  worse.  The  same  can  be  saia  for  using  3M 
data  only  versus  updating  the  forecast  with  alternative  data  when  3M  is  un¬ 
available  . 

The  smoothing  weight  and  data  concerns  can  be  handled  easily.  We  can  test 
which  forecast  performs  better  for  the  majority  of  the  items.  Therefore,  we 
compute  the  exponential  forecasts  with  3M  data  only  and  using  alternative  data 
when  3M  is  unavailable.  A  high  weight  and  a  low  weight  will  be  used.  We 
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chose  .4  and  .2  as  the  smoothing  weights,  because  under  current  policy,  .4  is 
used  at  the  consumer  level  and  2  at  the  wholesale  level. 

TRF  transitioning  and  minimizing  excessive  short  term  data  changes  are 
more  difficult  problems.  We  cannot  discard  the  TRF.  For  some  time  period, 
the  TRF  is  the  only  estimate  we  have  of  the  usage  rate.  Also,  Exponential 
Smoothing  must  have  observed  usage  to  identify  a  pattern  (an  average  or  level) 
and  project  that  pattern  into  the  future.  Furthermore,  we  need  to  have  obser¬ 
ved  some  usage  rates  to  determine  what  is  an  excessive  data  change. 

As  an  alternative  to  current  procedures,  we  adjust  the  TRF  over  the  de¬ 
velopment  period  with  a  probability  distribution.  We  can  treat  the  TRF  as  a 
point  estimate.  The  question  becomes:  given  this  estimate  of  the  usage  rate 
(TRF),  is  the  usage  rate  observed  realistic?  If  it  is,  then  the  TRF  is  a  rea¬ 
sonable  estimate  and  we  do  not  adjust  it.  Otherwise,  we  adjust  the  TRF  to  a 
more  reasonable  level.  We  selected  the  exponential  probability  distribution 
since  it  is  often  used  to  model  failure  rates,  has  been  shown  to  be  an  appro¬ 
priate  distribution  to  model  demand  (reference  2  of  APPENDIX  A) ,  and  requires 
only  one  estimated  parameter.  For  this  study,  the  development  period  was 
three  years.  We  set  a  lower  limit  based  on  what  we  would  expect  to  observe 
50%  of  the  time.  We  set  an  upper  limit  based  on  what  we  expect  to  observe  95% 
of  the  time.  For  the  first  three  years  (forecasts  2  through  4),  we  adjust  the 
TRF  if  the  usage  rates  observed  are  larger  or  smaller  than  we  would  expect, 
given  that  estimate  of  the  usage  rate.  In  the  fourth  year,  we  switch  to  Ex¬ 
ponential  Smoothing,  if  we  have  observed  four  years  of  data.  If  we  have  not, 
then  we  continue  to  adjust  the  TRF  until  we  have  observed  four  consecutive 
years  of  usage  data.  The  intention  here  is  to  develop  a  pattern.  The  de¬ 
velopment  period  should  be  long  enough  to  develop  a  pattern. 

With  Exponential  Smoothing,  (he  starting  point  can  he  critical;  therefore, 


we  considered  two  different  starting  points.  First,  we  used  the  adjusted  TRF 
described  above.  Since  the  TRF  has  been  adjusted  over  the  development  period, 
it  may  now  provide  a  valid  starting  point. 

For  the  other  starting  point,  we  used  an  average  of  the  first  three  years 
of  observed  usage  rates.  A  pattern  should  now  be  established  and  we  no  longer 
need  the  TRF.  However,  a  given  usage  rate  observation  may  be  distorted.  By 
using  an  average,  we  do  not  put  too  much  weight  on  one  observation;  i.e.,  we 
want  the  distortion  to  filter  out  quickly.  Starting  in  the  next  year  (the 
second  year  after  switching  to  Exponential  Smoothing) ,  we  "windsorize"  the 
most  recent  observation  using  the  probability  distribution  limits  previously 
used  to  adjust  the  TRF.  Here,  we  substitute  the  smoothed  forecast  in  place  of 
the  TRF.  That  is,  based  on  what  we  have  observed  in  the  past,  we  identify 
extreme  observe*-]  ns  for  the  usage  rate  as  those  that  fall  outside  upper  and 
lower  limi  .s  (This  method  was  developed  by  Tukey;  see  reference  3  of  APPEN¬ 
DIX  A.)  If  an  observation  is  outside  the  limits,  we  will  adjust  the  observa¬ 
tion  to  the  lower  or  upper  limit.  The  adjusted  usage  races  will  then  be 
smoothed  into  the  forecast.  Thus,  excessive  short  term  data  changes  will  be 
minimized. 

For  example,  a  BRF  in  year  7  with  a  value  of  1.0,  would  cause  the  8th 
year's  observed  usage  rate  to  be  constrained  between  .7  and  3.0.  Given  this 
constraint  on  the  usage  rate,  the  BRF  for  year  8  would  be  restricted  to  a 
value  between  .88  and  1.8  in  year  8.  We  want  to  find  or  establish  the  pattern 
of  the  data  (the  usage  rates).  Short  term  changes  distort  the  forecast  away 
from  the  true  pattern  of  the  data.  Therefore,  we  want  to  ignore  them.  How¬ 
ever,  we  can  not  distinguish  between  a  short  term  change,  the  pattern,  or  a 
new  emerging  pattern  until  after  the  fact. 

To  illustrate  this  procedure,  assume  an  item  has  the  usage  and  population 
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data  for  an  11  year  period  shown  in  TABLE  XI  and  a  TRF  of  2 .  Here  we  chose  a 
smoothing  weight  of  .4  and  the  average  data  starting  point  method. 


TABLE  XI 

SAMPLE  ALTERNATIVE  BRF  COMPUTATION 


YEAR 

USAGE 

POPULATION 

BRF  W/AVG  DATA 
STARTING  POINT 

1 

10 

1 

2 

2 

12 

2 

6 

3 

10 

2 

6 

4 

9 

3 

6 

5 

15 

5 

5.4 

6 

100 

2 

4.8 

7 

10 

2 

8.6 

8 

11 

1 

7.6 

9 

12 

2 

9.0 

10 

10 

5 

7.9 

11 

9 

3 

7.0 

The  BRFs  for  the  first  four  years  are  the  TRFs  adjusted  over  the  develop¬ 
ment  period  with  the  exponential  probability  distribution.  Since  we  have  four 
consecutive  years  of  observed  data,  the  remaining  BRFs  are  computed  using  ex¬ 
ponential  smoothing,  but  with  the  most  recent  observation  "Windsor ized"  to 
minimize  short  term  data  changes  using  the  probability  distribution  limits 
previously  used  to  adjust  the  TRF.  For  example,  the  usage  and  population  data 
in  year  6  are  used  to  compute  the  BRF  shown  in  year  7.  The  usage  rate  of  50 
(100/2)  for  year  6  is  excessive  compared  to  the  previous  years'  usage  rates. 
Thus,  we  "windsorize"  the  data  by  setting  the  year  6  usage  rate  to  14.4,  or 
three  times  the  BRF  (estimated  usage  rate  for  year  6).  Then  the  14.4  is 
smoothed  with  the  forecast  (4.8),  using  a  smoothing  weight  of  .4  to  get  the 
final  result  of  8.6  for  year  7. 
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We  have  now  identified  three  different  forecast  methods  with  various  para¬ 
meters  and  options,  for  a  total  of  11  alternatives  to  be  evaluated. 

•  Bayes  using  alternative  data  sources  when  3M  is  unavailable. 

•  Bayes  using  3M  data  only. 

•  Ratio  Method  using  3M  data  only. 

•  Exponential  Smoothing  using  alternative  data  sources  when  3M  is  un¬ 
available  . 


0 

a  —  .2 

average  data  starting  point 

o 

a  -  .4 

average  data  starting  point 

o 

a  —  .  2 

TRF  starting  point 

0 

a  »  .4 

TRF  starting  point 

Exponential  Smoothing  using  3M  data  only. 

O 

a  =  .  2 

average  data  starting  point 

O 

ci  —  .4 

average  data  starting  point 

0 

a  =  .  2 

TRF  starting  point 

o 

a  -  .4 

TRF  starting  point 

2.  Selecting  the  Best  Alternative.  To  begin  our  analysis,  we  first 
need  to  determine  the  TRF.  Therefore,  we  eliminated  (from  the  10  year  history 
file)  items  with  BRFs  in  the  oldest  year,  then  we  can  assume  that  the  first 

observed  BRF  is  the  item's  TRF.  To  obtain  meaningful  measurements  of  forecast 

performance,  items  also  had  to  have  at  least  five  years  of  history,  with  at 
least  three  years  of  the  history  containing  3M  data.  This  left  128,213  items 
to  measure  forecast  performance;  of  which,  91,543  were  identified  as  readiness 
contributors  by  their  FSC  codes  as  specified  by  NSLC. 

Since  there  are  a  large  number  of  items,  forecast  performance  was  eval¬ 

uated  based  on  criteria  which  measures  the  tendency  of  these  items  for  a  par¬ 
ticular  alternative.  Forecast  performance  was  ranked  based  on  that  criterion 
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from  one  for  best  performance  to  11  for  worst  performance.  Methods  which  tie 
for  a  particular  performance  element  were  assigned  the  average  of  the  ranks; 
e.g.,  two  methods  which  tie  for  best  performance  would  each  be  given  a  rank  of 
1.5  (1+2/2).  The  ranks  were  totaled  across  the  three  performance  measure¬ 
ments:  stability,  accuracy  with  respect  to  bias  and  extremes,  and  magnitude 

of  the  errors.  For  each  performance  measurement ,  we  used  five  different  sta¬ 
tistics.  Therefore,  no  one  performance  measurement  dominated  the  selection 
process.  Since  each  statistic  measured  the  performance  differently,  we  have  a 
fair  ’-epresentation  of  how  well  a  forecast  method  performs.  We  considered  the 
method  with  the  lowest  rank  sum  to  be  the  best  alternative. 

To  determine  how  the  alternative  methods  compare  to  the  current  procedure, 
we  computed  the  BRF  using  the  current  method  and  same  data  (TRFs  and  usage 
rates)  as  the  alternatives,  with  no  manual  changes,  ordnance  freezes,  or  over¬ 
rides.  This  is  the  Benchmark;  however,  we  did  not  consider  the  Benchmark  in 
the  ranking.  Ranks  shown  for  the  Benchmark  reflect  its  performance  relative 
to  the  performance  of  the  11  alternatives.  The  Benchmark  ranks  are  provided 
only  to  facilitate  comparisons;  i.e.,  to  determine  whether  or  not  the  alter¬ 
natives  provide  better  results  than  the  current  procedure.  For  example,  the 
Benchmark's  rank  was  set  equal  to  an  alternative's  rank  when  their  numbers 
(mean,  median,  etc.)  were  the  same. 

a.  Stability .  We  defined  stability  based  on  a  method's  ability 
to  forecast  within  the  95%  confidence  limits  of  the  average  3M  usage  rate. 
Therefore,  we  used  the  number  of  items  always  within  these  limits  as  a  perfor¬ 
mance  element.  The  method  with  the  most  items  has  the  best  performance,  the 
least  items  has  the  worst  performance.  We  also  measured  the  mean  number  of 
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times  items  were  outside  the  limits.  A  forecast  with  a  large  mean  has  a  high¬ 
er  number  of  items  outside  the  limits  for  more  years  than  one  with  a  smaller 
mean.  Therefore,  we  assigned  the  best  (lowest)  rank  to  the  method  with  the 
smallest  mean  and  the  highest  (worst)  rank  to  the  largest  mean.  The  median  is 
not  influenced  by  large  or  small  values.  Therefore,  a  forecast  with  a  small¬ 
er  median  than  another  method  is  more  stable.  The  mode  is  the  most  frequent 
observation.  We  assigned  the  best  rank  to  the  alternative  with  the  largest 
number  of  items  representing  the  smallest  mode.  Finally,  we  considered  the 
forecast  alternative  with  the  smallest  standard  deviation  to  be  the  most 
stable,  since  it  is  not  as  widely  dispersed  as  the  other  methods  (has  more 
values  closer  to  the  mean) . 

The  stability  results  are  displayed  for  the  readiness  contributors  in 
TABLE  XII  and  for  all  items  in  TABLE  XIII.  The  results  are  consistent  for 
both  data  sets.  The  lowest  cumulative  ranks  were  achieved  by  the  Exponential 
Smoothing  methods  using  alternative  data  sources  and  a  data  starting  point 
with  either  weight.  These  two  alternatives  were  also  ranked  top  in  each  cate¬ 
gory  with  the  exception  of  the  "NO.  ITEMS  ALWAYS  W/I  LIMITS"  category. 
But  even  there,  the  numerical  differences  were  small.  However,  the  overall 
rank  difference  between  these  two  most  stable  alternatives  and  the  methods 
available  under  Resystemization  (Ratio  and  Benchmark)  are  largo. 

Forecasts  using  alternative  data  when  3M  data  was  unavailable  produced 
more  stable  forecasts.  For  the  Bayes  forecasts,  using  alternative  data  re¬ 
sulted  in  a  few  more  items  always  within  the  limits  (648  for  readiness  con¬ 
tributors/821  for  all  items).  This  effect  was  more  pronounced  for  the  Expo¬ 
nential  Smoothing  forecasts  than  the  Bayes.  With  Exponential  Smoothing 
(average  data  starting  point  either  weight),  the  difference  was  primarily  in 
the  mode.  The  value  for  the  mode  was  the  same,  but  over  2.3  times  as  many 
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items  had  that  value  for  the  alternative  data  forecast  vs  the  3M  data  only 
forecast.  Consequently,  the  means  and  standard  deviations  are  less  for  the 
alternative  data  forecast;  i.e.,  using  only  3M  data  produced  forecasts  that 
were  outside  of  the  limits  for  more  years  than  using  alternative  data  to  up¬ 
date  the  forecast  when  3M  was  unavailable. 

For  Exponential  Smoothing,  the  average  data  starting  point  produced  more 
stable  forecasts  than  the  TRF  starting  point  regardless  of  the  data  source  or 
weight.  The  means,  medians,  modes,  and  standard  deviations  are  higher  for  any 
of  the  TRF  starting  point  forecasts  than  the  data  starting  point  forecasts. 
(Note  the  median  and  modes  for  the  data  starting  point  forecasts  are  three 
years.  For  all  other  methods,  the  median  and  modes  are  six  years.  Since  we 
discarded  the  first  forecast,  we  would  have  used  the  adjusted  TRF  for  a  mini¬ 
mum  of  three  years.) 

We  used  ranks  in  order  to  make  an  objective  decision  with  multiple  cri¬ 
teria.  A  problem  with  ranking  is  that  small  differences  may  be  inflated  and 
large  differences  may  appear  small.  Consequently,  care  was  taken  that  no  one 
category  or  statistic  would  dominate  the  selection  process.  To  insure  that 
this  occurred  and  that  one  rank  did  not  have  an  undue  influence,  we  reviewed 
each  statistic,  comparing  the  selected  method  with  the  alternatives.  With 
respect  to  stability  (TABLES  XII  and  XIII),  no  method  performed  better  than 
Exponential  Smoothing  using  alternative  data  sources  and  an  average  data 
starting  point  based  on  the  mean,  median,  mode,  and  standard  deviation.  The 
Bayes  and  Ratio  alternatives  did  forecast  within  the  limits  for  more  items 
over  the  entire  forecast  horizon  than  Exponential  Smoothing.  However,  the 
maximum  difference  was  less  than  3%  of  the  universe. 


t 
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STABILITY  FINDIN6S 
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BENCHMARK  RANKS  ONLY  DENOTE  RANKING  RELATIVE  TO  ALTERNATIVE  FORECASTS. 
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BENCHMARK  RANKS  ONLY  DENOTE  RANKING  RELATIVE  TO  ALTERNATIVE  FORECASTS. 


b.  Bias  and  Extremes .  Negative  MEs  denote  over  forecasts  (less 


risk  of  stockout).  Therefore,  we  assigned  the  method(s)  with  the  least  nega¬ 
tive  mean  and  median  MEs  (i.e,  the  least  overforecast)  the  best  (lowest)  ranks 
and  the  method  with  the  most  positive  mean  and  median  MEs  (i.e.,  the  greatest 
underforecast)  the  worst  (highest)  ranks.  All  overforecasts  were  ranked 
better  than  the  underforecasts.  Since  one  extreme  value  can  influence  the 
mean  and  we  are  interested  in  how  well  the  forecast  methods  handle  extreme 
values,  the  method(s)  with  the  smallest  (in  absolute  terms)  maximum  and  mini¬ 
mum  were  considered  as  performing  the  best.  The  standard  deviation  also  de¬ 
notes  how  well  a  method  handles  extreme  values.  A  method  with  a  smaller  stan¬ 
dard  deviation  has  less  extreme  values  than  one  with  a  larger  standard  devia¬ 
tion. 

The  results  with  respect  to  bias  and  extremes  are  displayed  for  the  read- 
n ess  contributors  in  TABLE  XIV  and  all  items  in  TABLE  XV.  Here,  the  results 
are  inconsistent  between  the  readiness  contributors  and  all  items.  Based  on 
readiness  contributors,  the  Exponential  Smoothing  (.A  smoothing  weight)  using 
alternative  data  with  either  starting  point  are  the  best  performers  with  cumu¬ 
lative  ranks  of  about  16.  The  current  BRF  method  is  next  at  19.  Across  all 
items,  all  the  methods  except  smoothing  with  the  TRF  starting  point  are  ranked 
fairly  close. 

The  results  were  also  inconsistent  between  readiness  contributors  and  all 
items  for  3M  only  vs  alternative  data  forecasts.  For  all  items,  for  both  the 
Bayes  and  Exponential  Smoothing  forecasts  (all  weights  and  starting  points), 
the  3M  ot  <.y  data  forecasts  outperformed  the  alternative  data  forecasts.  For 
the  readiness  contributors,  the  alternative  data  outperformed  the  3M  data  only 
forecasts.  The  average  data  starting  point  performed  better  for  all  items; 
while  there  was  little  difference  between  the  two  starting  points  for  the 
readiness  contributors. 
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With  respect  to  bias,  the  smoothing  method  (.4  weight)  with  alternative 
data  sources  and  average  data  starting  point  method  does  have  a  tendency  to 
err  by  overforecasting.  However,  it  does  not  overforecast  as  much  as  other 
alternatives.  The  medians  were  the  second  best  for  both  the  readiness  con¬ 
tributors  and  all  items.  Furthermore,  the  mean  performed  second  best  for  the 
readiness  contributors  and  third  best  for  all  items.  In  terms  of  handling  ex¬ 
treme  values,  this  method  was  ranked  near  the  middle.  Compared  to  the  methods 
available  under  Resystemization ,  this  method  does  not  overforecast  as  much  as 
the  Benchmark  and  it  does  not  handle  extreme  forecast  errors  as  well.  The 
Ratio  Method  consistently  under forecasts  based  on  the  medians  and  means. 
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BENCHMARK  RANKS  ONLY  DENOTE  RANKING  RELATIVE  TO  ALTERNATIVE  FORECASTS. 


c.  Accuracy .  Small  MSEs  indicate  errors  that  are  small  in  mag¬ 
nitude.  Therefore,  we  considered  the  alternal lve(s)  with  the  smallest  mean 
and  median  MSEs  to  be  the  most  accurate  forecast.  Since  we  have  already  con¬ 
sidered  extreme  values,  we  now  focus  on  the  upper  and  lower  quartiles.  These 
statistics  are  similar  to  the  median  and  insensitive  to  extreme  values.  The 
upper  quartile  has  25%  of  the  observations  greater  than  or  equal  and  75%  less 
than  or  equal  to  it.  The  lower  quartile  has  75%  of  the  observations  greater 
than  or  equal  and  25%  of  the  observations  less  than  or  equal  to  it.  They 
allow  us  to  ignore  one-fourth  of  the  largest  values  and  one-fourth  of  the 
smallest  values.  We  assign  the  method  with  the  smallest  MSE  quartile  the  best 
(lowest)  rank.  We  use  the  standard  deviation  here  also.  Again,  we  consider 
the  method  with  the  smallest  value  for  the  standard  deviation  to  be  the  best 
performer . 

The  results  for  accuracy  with  respect  to  the  magnitude  of  the  errors  are 
displayed  in  TABLE  XVI  for  the  readiness  contributors  and  TABLE  XVII  for  all 
items.  The  MSEs  will  be  small  for  the  quartiles  and  medians.  (This  is  the 
nature  of  the  measurements.  We  would  be  concerned  if  they  were  not,  given  the 
small  size  of  most  BRFs  and  usage  rates.) 

The  tables  show  that  for  readiness  contributors  and  for  all  items,  the  Ex¬ 
ponential  Smoothing  methods  using  alternative  data  and  the  TRF  starting  point 
(either  weight),  and  3M  only  with  a  weight  of  .4  and  the  TRF  starting  point 
are  the  best  performers.  For  readiness  contributors,  the  ranks  were  15-17.5. 
The  Exponential  Smoothing  method  using  alternative  data  sources,  an  average 
data  starting  point  with  a  weight  of  .4  is  close  with  a  total  rank  of  22. 
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BENCHMARK  RANKS  ONLY  DENOTE  RANKING  RELATIVE  TO  ALTERNATIVE  FORECASTS 


Based  on  the  quartiles  and  medians,  using  alternative  data  when  3M  data 
was  unavailable  produced  more  accurate  forecasts  for  all  items  and  the  readi¬ 
ness  contributors  regardless  of  method.  The  means  and  standard  deviations 
were  usually  slightly  smaller  for  the  3M  only  data  forecasts  than  the  alter¬ 
native  data  forecasts.  This  is  consistent  with  our  ME  findings.  The  3M  only 
data  forecasts  tended  to  perform  better  with  respect  to  extreme  values  than 
the  alternative  data  forecasts.  Remember,  the  mean  and  standard  deviation  are 
influenced  by  extreme  values;  the  quartiles  and  medians  are  noc. 

For  Exponential  Smoothing  using  the  TRF  as  a  starting  print  produced  more 
accurate  forcasts  than  using  an  average  data  starting  point.  The  TRF  starting 
point  using  alternative  data  with  a  smoothing  weight  of  .2  tended  to  be  the 
most  accurate  of  the  forecast  methods  (with  respect  to  the  magnitude  of  the 
errors).  All  the  Exponential  Smoothing  methods  performed  better  with  respect 
to  accuracy  for  the  readiness  contributors  than  all  items. 

d.  Overall  Performance.  To  determine  the  best  forecast  with 
respect  to  stability,  bias  and  extremes,  and  magnitude  of  errors,  we  totaled 
the  ranks  across  each  category.  The  rank  sums  are  displayed  in  TABLE  XVIII 
for  the  readiness  contributors  and  TABLE  XIX  fcr  all  items.  Based  on  the 
total  ranks,  Exponential  Smoothing  with  a  data  windsorizing  system,  a  smooth¬ 
ing  weight  of  .4,  an  average  data  starting  point,  and  alternative  data  sources 
when  3M  data  is  unavailable,  provides  the  best  forecast  for  readiness  contri¬ 
butors.  For  all  items,  it  is  the  second  best  alternative,  but  only  by  one 
rank  (the  difference  between  the  best  and  second  best  alternative  methods  for 
all  items  is  the  smoothing  weight) . 


TABLE  XVIII 
FINDINGS 

READINESS  CONTRIBUTORS  ONLY  (91.543) 
GRAND  TOTAL  RANKS 


TOTAL  RANKS 

STABILITY 

ME 

MSE 

TOTAL 

BAYES  ALT  DATA 

30 

37 

44.5 

111.5 

BAYES  3M  ONLY 

33.5 

43 

50 

126.5 

RATIO  3M  ONLY 
EXPONENTIAL  SMOOTHING: 

39 

34.5 

46 

119.5 

ALT  DATA: 

a  =  .2  DATA  ST  PT 

11 

23.5 

27 

61.5 

a  -  .4  DATA  ST  PT 

13 

16 

22 

51 

a  =  .2  TRF  ST  PT 

36 

24 

15 

75 

a  -  .4  TRF  ST  PT 

36.5 

15.5 

17.5 

69.5 

3M  ONLY: 

a  —  .2  DATA  ST  PT 

21 

34 

40 

95 

a  -  .4  DATA  ST  PT 

23 

33 

32.5 

88.5 

a  -  .2  TRF  ST  PT 

40 

37.5 

28 

105.5 

a  —  .4  TRF  ST  PT 

37 

32 

17.5 

86.5 

*BENCHMARK 

41.5 

mm 

29.5 

90.0 

*NOTE:  BENCHMARK  RANKS  ONLY  DENOTE  RANKING  RELATIVE  TO  ALTERNATIVE 

FORECASTS 
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TABLE  XIX 
FINDINGS 

ALL  ITEMS  (128 .215) 
GRAND  TOTAL  RANKS 


TOTAL  RANKS 

STABILITY 

ME 

MSE 

TOTAL 

BAYES  ALT  DATA 

33.5 

27 

26.5 

87 

BAYES  3M  ONLY 

32 

24 

32 

88 

RATIO  3M  ONLY 
EXPONENTIAL  SMOOTHING: 

40.5 

27 

39 

106.5 

ALT  DATA: 

a  =  .2  DATA  ST  PT 

12 

27.5 

30.5 

70 

a  =  .4  DATA  ST  PT 

15 

29.5 

26.5 

71 

a  =  .2  TRF  ST  PT 

38 

39.5 

20.5 

98 

a  -  .4  TRF  ST  PT 

37.5 

34.5 

21.5 

93.5 

3M  ONLY: 

a  -  .2  DATA  ST  PT 

19.5 

21.5 

43 

84 

a  -  .4  DATA  ST  PT 

22.5 

28.5 

37.5 

88.5 

a  -  .2  TRF  ST  PT 

41 

38.5 

31.5 

111 

a  -=  .4  TRF  ST  PT 

38.5 

32.5 

21.5 

92.5 

*BENCHMARK 

44 

24 

30 

98 

*NOTE:  BENCHMARK  RANKS  ONLY  DENOTE  RANKING  RELATIVE  TO  ALTERNATIVE 

FORECASTS 


Therefore,  we  conclude  that  Exponential  Smoothing  with  a  data  Windsor izing 
system,  alternative  data  when  3M  data  is  unavailable,  an  average  data  starting 
point,  and  a  .4  smoothing  weight  is  the  best  alternative  with  respect  to  sta¬ 
bility,  accuracy  with  respect  to  bias  and  extremes,  and  the  magnitude  of 
errors.  That  is  not  to  say  it  is  the  most  stable  or  accurate  of  all  the  al¬ 
ternatives,  but  that  method  provides  the  best  combination  of  stability  and 
accuracy . 

We  also  looked  at  the  impact  on  two  equipments  (the  engine  and  PHAIANX) 
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using  the  same  performance  measurements.  The  detailed  ;tati sties  for  the 


equipments  are  found  in  APPENDIX  E.  The  results  are  very  close  across  the  al¬ 
ternatives  for  both  equipments.  The  proposed  method  provided  slightly  better 
results  with  respect  to  stability  and  accuracy  for  the  engine.  The  PHALANX 
results  were  inconclusive. 

Since  Depot  Level  Repairable  (DLR)  items  constitute  the  major  portion  of 
the  COSAL  cost,  we  looked  at  the  performance  measurements  with  respect  to 
these  items.  The  grand  total  ranks  are  displayed  in  TABLE  XX.  Again,  the 
proposed  method  provided  the  best  results  with  respect  to  stability,  bias  and 
extremes,  and  accuracy.  The  methods  available  under  resystemization  (the 
current  and  Ratio  methods)  performed  the  worst. 

TABLE  XX 

DLR  ITEMS  (6.605) 

GRAND  TOTAL  RANKS 


TOTAL  RANKS 

STABILITY 

ME 

MSE 

TOTAL 

BAYES  ALT  DATA 

25 

39.5 

36.5 

101 

BAYES  3M  ONLY 

27 

34.5 

41 

102.5 

RATIO 

EXPONENTIAL  SMOOTHING: 

42 

23 

49 

114 

ALT  DATA: 

a  -  .2  DATA  ST  PT 

21 

26 

31 . 5 

78 . 5 

a  -  .4  DATA  ST  PT 

24 

19.5 

28.5 

72 

a  -  .2  TRF  ST  PT 

33.5 

19.5 

22 

75 

a  -  .4  TRF  ST  PT 

34  5 

22.5 

18 

75 

3M  ONLY: 

a  -  .2  DATA  ST  PT 

25 

30 

31 .  5 

86.5 

a  -  .4  DATA  ST  PT 

24 

34 

28 

86 

a  -  .2  TRF  ST  PT 

34.5 

41 

21 

96.5 

a  -  .4  TRF  ST  PT 

34.5 

41 

18 

93.5 

★BENCHMARK 

42.5 

31 

35 

108.5 

★NOTE :  BENCHMARK  RANKS  ONLY  DENOTE  RANKING  REIATIVE  TO  ALTERNATIVE  FORECASTS 
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3.  Impact  of  the  Proposed  Change.  The  impact  of  implementing  the 


recommended  forecasting  method  was  measured  in  terms  of  COSAL  cost  and  effec¬ 
tiveness.  Measuring  the  impact  was  restricted  due  to  the  following  limita¬ 
tions.  First,  we  do  not  know  the  items'  TRFs.  To  determine  the  best  fore¬ 
casting  method,  we  used  items  with  less  than  10  years  of  BRF  history.  There¬ 
fore,  we  felt  confident  that  an  item's  first  BRF  was  its  TRF.  In  this  impact 
analysis,  the  item  can  have  more  than  10  years  of  history.  The  first  BRF  is 

not  necessarily  the  TRF.  In  those  cases,  the  TRF  is  now  smoothed  out  and  the 

transitioning  effect  is  lost.  However,  we  have  no  choice  but  to  treat  an 
item's  first  BRF  as  the  TRF.  Secondly,  little  data  was  available  to  measure 
COSAL  effectiveness  over  time.  The  COSAL  candidate  files  were  as  of  July 
1988;  consequently,  little  subsequent  demand  data  was  available.  Therefore, 
we  could  not  determine  the  impact  of  long  term  vs  short  term  benefits,  especi¬ 
ally  with  respect  to  COSAL  stability.  Furthermore,  with  a  small  number  of 
item  demands,  one  or  two  demands  can  distort  the  effectiveness  statistics. 

The  relevant  COSAL  statistics  for  the  six  ships  are  displayed  in  TABLEs 

XXI  through  XXIII.  The  Benchmark  (BM)  is  the  current  method.  As  in  the 

previous  section,  we  computed  the  Benchmark  with  the  same  data  (TRF  and  usage 
rates)  as  the  alternative.  Manual  changes,  ordnance  freezes,  and  overrides 
are  removed.  Therefore,  we  can  compare  methods.  The  system  BRF  (SYS)  is  the 
one  on  file.  This  includes  manual  changes,  ordnance  freezes,  and  overrides, 
and  it  may  be  computed  with  data  not  on  the  history  file.  The  proposed  method 
(Exponential  Smoothing  with  a  data  windsorizing  system,  a  smoothing  weight  of 
.4,  alternative  data  when  3M  data  is  unavailable,  and  an  average  data  starting 
point)  is  the  alternative  (ALT).  Range  and  cost  statistics  are  displayed  in 
TABLE  XXI.  The  effectiveness  statistics  for  all  items  are  displayed  in  TABLE 
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XXII  and  for  the  readiness  contributors  in  TABLE  XXIII.  Generally,  the  cost 
is  about  the  same  with  the  alternative  as  the  Benchmark,  while  requisition 
effectiveness  for  readiness  contributors  decreased  by  0.0  to  2.8  percentage 
points  across  the  test  ships.  Remember,  however,  that  these  effectiveness 
numbers  only  represent  a  handful  of  demands.  With  the  alternative,  there  is  a 
slight  (3%)  decrease  in  range.  Both  the  Benchmark  and  the  alternative  have 
less  range  and  effectiveness  than  the  system  file  BRF. 

4.  Conclusions  -  BRF  Improvement.  We  found  that  Exponential  Smooth¬ 
ing  with  a  data  windsorizing  system,  a  smoothing  weight  of  .4,  alternative 
data  sources  when  3M  data  is  unavailable,  and  an  average  data  starting  point 
provides  the  best  forecast  in  terms  of  stability,  accuracy  with  respect  to 
bias  and  extremes,  and  magnitude  of  the  errors.  This  finding  was  consistent 
for  all  items  and  the  readiness  contributors.  These  rates  resulted  in  test 
COSALs  with  3%  less  range,  the  same  cost,  and  reduced  effectiveness  0.0  to 
2.8  percentage  points  for  readiness  contributors.  These  COSAL  statistics  are 
inconclusive  since  we  had  limited  data  and  only  a  few  items  had  demand.  Fur¬ 
thermore,  we  do  not  know  what  manual  overrides  might  be  forced  on  the  proposed 
method. 
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FINDINGS 
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IV.  CONCLUSIONS 


BRF  instability  and  churn  are  primarily  related  to  the  subjective  portion 
of  the  forecast  (TRFs,  ordnance  freezes,  manual  changes  and  overrides).  Eli¬ 
minating  the  TRF,  ordnance  freezes,  manual  changes  and  overrides  from  the 
forecast  produced  more  stable  forecasts  that  trended  close  to  the  actual  3M 
average  usage  rates  computed  from  all  available  data. 

The  proposed  alternative  BRF  computation  (Exponential  Smoothing  with  a 
data  windsorizing  system,  a  smoothing  weight  of  .4,  alternative  data  sources 
when  3M  data  was  unavailable,  and  an  average  data  starting  point)  provides  the 
best  forecast  in  terms  of  stability,  accuracy  with  respect  to  bias  and  ex¬ 
tremes,  and  the  magnitude  of  the  errors.  These  results  were  generally  consis¬ 
tent  for  all  items  and  the  readiness  contributors.  Although  the  proposed  com¬ 
putation  appears  to  slightly  reduce  COSAL  range  and  effectiveness,  we  believe 
the  improved  stability  and  accuracy  will  provide  long  term  benefits  in  reduced 
COSAL  churn  which  tends  to  create  long  supply  and/or  outfitting  deficiencies. 

V.  RECOMMENDATION 

We  recommend  using  the  exponential  probability  distribution  to  adjust  the 
TRF  during  the  development  period.  After  the  development  period,  compute  the 
BRF  using  the  Exponential  Smoothing  method  with  a  smoothing  weight  of  .4,  a 
data  windsorizing  system  to  handle  extremes,  an  average  data  starting  point, 
and  alternative  data  when  3M  data  is  unavailable. 
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APPENDIX  B;  FORECAST  METHODS 


The  following  paragraphs  describe  the  four  forecast  methods  discussed  in 
this  study  (Current  Method,  Ratio  Method,  Bayes  Method,  and  Exponential 
Smoothing  using  the  exponential  probability  distribution  to  adjust  the  Tech¬ 
nical  Replacement  Forecast  (TRF)  during  the  development  period  and  a  data 
windsorizing  system)  and  show  examples  of  each. 

1 .  Current  Method  (Benchmark) . 

The  current  method  has  a  two  year  development  period.  That  is,  the 
TRF  is  used  as  the  forecast  until  over  two  full  years  of  observed  usage  is 
available.  (The  development  period  is  a  parameter.)  For  the  third  forecast 
of  the  Best  Replacement  Factor  (BRF),  the  usage  rate  is  .smoothed  into  the  TRF 
with  a  .4  smoothing  weight.  The  preferred  data  source  is  Navy  Maintenance  and 
Material  Management  (3M);  but  if  3M  data  is  unavailable,  then  Mobile  Logistics 
Support  Force  (MLSF)  data  is  used.  When  both  3M  and  MLSF  data  are  unavail¬ 
able,  system  data  is  used.  If  no  data  is  available,  the  BRF  is  not  updated. 


Year 

1  &  2: 

BRF1  -  BRF2  = 

TRF 

Year 

3: 

BRF3  =  a  X2  + 

(1-a) 

TRF 

Year 

4: 

BRF*  =•  a  X3  + 

(1-a) 

BRF3 

Year 

T+l : 

BRFTtl  —  a  Xy 

+  (1-a 

)  BRF. 

where 

XT  -  the  usage  rate  for  the  Tth  period  (demand  for  the  Tth  period  divided 
by  average  population  for  the  Tth  period 
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2 .  Ratio  Method. 

The  Ratio  Method  has  at  least  a  two  year  development  period;  that  is, 
the  first  two  BRFs  are  the  TRF.  The  development  period  continues  if  the  item 
neither  had  two  demands  nor  expected  to  see  two  demands  by  this  point  in  time. 
When  two  demands  have  been  observed,  then  the  BRF  becomes  the  sum  of  the  de¬ 
mands  divided  by  the  sum  of  the  population.  If  two  demands  have  not  been  ob¬ 
served,  but  have  been  expected,  the  BRF  becomes  the  inverse  of  the  sum  of 
population.  This  method  presently  uses  only  3M  data  (but  could  be  modified  to 
consider  other  data) . 

Year  1  &  2:  BRFj  =  BRF2  =  TRF 

Thereafter: 

T 

If  cumulative  demand  is  two  or  greater,  i.e.,  2  Di  >  2, 

i-1 

T 

2  Dt 

.  „„„  i=l 

then  BRFt+1  -  — - - 

S  Pi 
1=1 


If  cumulative  demand  is  less  than  two,  but  expected  cumulative  demand  is 


two  or  greater;  i.e. 


T 

2  Da  <  2  and  TRF 
i=l 


U 


>  2, 


then  BRFTtl  = - ; 

2  P, 

i=l 


otherwise,  BRFT+1  »  BRFT. 
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to  Adjust  the  TRF  during  the  Development  Period  and  a  Data  Windsorizine  Svs 


Here,  the  demand  development  period  is  again  a  parameter.  For  this 
study,  we  will  use  the  three  year  development  period  proposed  for  Resyst.omi r.a - 
tion.  During  that  time,  we  will  adjust  the  TRF  using  the  exponential  probabi¬ 
lity  distribution.  We  compare  the  actual  usage  rate  to  that  expected  based  on 
the  TRF.  If  the  actual  usage  rate  is  within  user  established  control  bands, 
the  TRF  remains  unchanged.  If  the  actual  usage  rate  is  outside  one  of  these 
bands,  the  TRF  is  set  to  the  applicable  bound. 

For  the  exponential  distribution: 


F(X)  =  1  -  e 
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where 


F(X)  -  the  reverse  cumulative  distribution 
e  -  the  natural  log 
Xx  =  the  usage  rate 

For  F(X) ,  we  chose  an  upper  bound  of  .95  and  a  lower  bound  of  .5.  There¬ 
fore  , 


F(X)  =  .95  =  1  -  e  BRFl  -  Max  Xx  =  BRFX  *  3 

.  ?1__ 

RRF 

F (X)  =  .50  =  1  -  e  1  -  Min  Xt  =  BRFX  *  .7 

The  BRF  is  then  computed  as  follows: 

Year  1:  BRFX  =  TRF 
Thereafter : 

If  BRFl  *  .7  <  Xx  <  BRFX  *  3,  then  BRF2  =  BRFX. 

If  Xx  >  BRFX  *  3,  then  BRF2  *  BRFX  *  3. 

If  Xx  <  BRFX  *  .7,  then  BRF2  =  BRFX  *  .7. 

BRF3  and  BRF4  are  computed  the  same  way  as  BRFZ  using  the  appropriate  usage 
rate  (X2  and  X3)  and  BRF  (BRFZ  and  BRF3)  .  The  idea  here  is  that  we  treat  the 
TRF  as  a  reasonable  estimate  of  the  usage  rate  as  long  as  we  observe  usage 
rates  that  we  would  expect  based  on  that  estimate.  However,  we  are  not  com¬ 
pletely  sure  of  our  data.  Therefore,  we  will  adjust  our  TRF  to  a  reasonable 
bound  when  the  observations  are  outside  the  bounds.  After  four  years,  we 
should  have  developed  a  reliable  source  of  history.  Therefore,  if  we  have 
observed  four  years  of  usage  rates,  we  will  switch  to  Exponential  Smoothing. 
Otherwise,  we  will  continue  to  adjust  the  BRF  as  for  the  first  four  forecasts 
until  we  observe  four  consecutive  years  of  usage  rates  (i.e.,  four  consecutive 


years  of  population  greater  than  0  indicating  there  was  a  potential  for 
demand;  the  demand  observation  could  be  0).  There  are  two  alternatives  for 
starting  points.  One,  we  can  average  the  first  three  years  of  usage  rates  and 
smooth  in  the  fourth  years  usage  rate.  Two,  we  can  use  the  adjusted  TRF. 

•  Average  Data . 


brf5 


aX4  +  (1-a) 


'  3 

2  X± 
1=1 

3 


•  Adjusted  TRF. 

BRF5  =  aX*  +  (1-a)  [aX3  +  (l-a)(aX2  +  (1-aHaX!  +  (1-a)  BRF*))] 

Now  that  we  have  developed  a  history  of  usage  rates,  we  can  define  what  is  a 
distorted  observation.  Observations  that  are  distorted  can  be  "windsorized" 
using  the  exponential  distribution.  That  is,  we  do  not  allow  abnormally  small 
or  large  values  to  distort  the  forecast.  Abnormal  values  are  based  upon  past 
observations . 


For  i  >  5,  when  i  =  1,  2,  3,  ...  T: 

If  XA  >  BRFi  *  3,  then  we  set  X*  =  BRFj  *  3. 

If  Xt  <  BRFt  *  .7,  then  we  set  Xj  =  BRFt  *  .7. 


5 .  Examples  . 

a.  Example  1 .  For  this  example,  we  will  assume  the  data  is  all  3M , 
the  TRF  is  .1,  and  the  smoothing  weight  is  .4.  The  following  table  demonstra¬ 
tes  how  the  alternate  methods  forecast  usage  rates  for  an  item  with  zero  ob¬ 
served  usage . 
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RATIO 

BAYES 

EX  SMOOTHING/ 
AVG  DATA 

EX  SMOOTHING/ 
ADJUSTED  TRF 

.1 

.1 

.1 

.1 

.009 

.07 

.005 

.005 

.049 

.003 

.003 

Wmmm 

.034 

.002 

.002 

0 

.004 

.002 

.002 

0 

.004 

.001 

.002 

0 

.003 

.001 

.002 

0 

.003 

.001 

.001 

0 

.003 

.001 

.001 

0 

.002 

.001 

.001 

0 

.001 

Note  that  the  average  data  starting  point  forecast  is  the  only  one  that  reach¬ 
es  zero.  The  others  only  approach  zero. 

b.  Example  2.  For  this  example,  we  will  assume  the  data  is  all  3M, 
the  TRF  is  2,  and  the  smoothing  weight  for  the  average  data  starting  point  is 
.4  and  .2  for  the  adjusted  TRF  starting  point.  The  following  table  demonstra¬ 
tes  how  the  alternatives  forecast  items  with  demand  in  every  year,  including 
one  year  (year  6)  with  obvious  extreme  values. 


EX  SMOOTHING/ 

EX  SMOOTHING/ 

AVG  DATA 

ADJUSTED  TRF  1 

i 

2 

2 

6 

6 

6 

6 

6  1 

6 

5.4 

5.6 

4.8 

5.3 

8.6 

7.4 

7.6 

7.0 

9.0 

7.8 

7.9 

7.4 

7.0 

7.0 

The  average  data  starting  point  and  adjusted  TRF  starting  point  forecasts  do 
not  react  as  strongly  to  the  extreme  usage  rate  value  of  50  in  year  6. 
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APPENDIX  C:  FORECAST  WEIGHTS 


In  this  section  we  develop  mathematically  the  weights  for  more  recent  ob¬ 
servations  versus  older  observations. 

1 .  Exponential  Smoothing . 

BRFTtl  =  XT 


brft+2 

-  aXT+1 

+ 

(1-a) 

BRFTtl 

=  aXTtl 

+ 

(1-a) 

xT 

• 

BRFt+3 

=  qXt+2 

+ 

(1-a) 

BRFt+2 

=  aXT+2 

+ 

(1-a) 

aXT+1  + 

(1-a)2 

xT 

BRFm 

”  qXT+3 

+ 

(1-a) 

brft+3 

“  aXT+3 

+ 

(1-a) 

aXT+2  + 

(1-a)2 

BRFt+„  -  oXT(H_1  +  (1-a)  BRFt+h_1  - 

qXt+R-1  +  (l"a)  qXT+H-2  +  (l"a)Z  qXt+M-3  +  •  •  •  +  (l*a)N  1  XT 


where 

BRFT  *  the  forecast  for  the  Tth  period,  T  =  1,  2,  3,  ...  T 
XT  -  the  usage  rate  (or  TRF)  for  the  Tth  period 
a  =  smoothing  weight 
2 .  Ratio  Method. 


T 

2  D4 

BRFt.i  =  - - 

>-  P, 
i=l 


Pi  •(-  P2  +  P3  +  ....  PT 
T 

S  P, 
i=l 


1 


i=l 


Pi 


(Di)  +  (D2)  +  ~Y - <d3)  +  .  .  .  +  -----  (Dt) 

2  Pi  2  Pi  2  Pt 

i=l  i=l  i=l 
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_ 1_ 

T 

S  P, 
1=1 


+ 


__1_ 

'  T 
2  Pt 
1=1 


<D2)  + 


1 

t" 

2  P4 

i=l 


(03)  4 


T 

2  Pi 
1=1 


Substituting  Xt  for  Dj/Pj  (the  usage  rate)  and  simplifying  yields: 


j-A~  (Xi)  + 


(X2)  +  (X3)  + 


■  +  -f-r-  (XT) 

2  Pi 

1=1 


where 

BRFt  =  the  forecast  for  the  Tth  period,  T  =  1,  2,  3,  ...  T 
Dt  =  the  3M  demand  for  the  Ith  period 
PT  =  the  average  3M  population  for  the  Tth  period 
If  the  population  sizes  are  constant,  then  Pj  =  P2  =  P3  =  ...  -  PT 


Fp  <Xj)  +  --  (X2)  +  --  (X3)  +  . 


■  (XT) 


\  (Xx)  +  \  (X2)  +  \  (X3)  +  . 


•  +  *  (X,) 


If  the  population  sizes  double  each  year,  then 


2T~1P1  =  2t  zP2  =  2T  3P3  =  .  .  .  =  2Pt_j  =  PT 


?T..  =  (*i>  +  T  i  (x2>  +  T-i  '  (X;}) 

2  2 ‘P,  2  2‘Pi  2  2‘P, 


2T  *P 

+  •  •  •  +  <-Xt) 

2  2  ‘P, 

i=0 
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T-  1  +  rp  (X2)  +  ,p_“p  (X3) 

2  21  2  21  2  2l 

i=0  i=0  i=0 


+  .  .  .  + 


T-l 

2  2 ' 
i=0 


If  the  population  sizes  halve  each  year,  then 


Pi  =  2P2  =  22P3  =  .  .  .  =  2T'2PT_1  =  2t’1Pt 


2t"1P1 


2t-3Pi 


“I —  <xi>  +  ~yt~  Cx2>  +  — T-rf-  <x3)  + 


2  2  P, 


2  2‘Pj 

i=0 


2  2*P, 


T-l 

2  2‘P, 

i=0 


2t-i 

~y.t~  <xi>  + 
2  21 
i=0 


2T-2  2t-3 

(xz)  +  Y"-"X  (X3)  + 

2  21  2  21 

1=0  i=0 


T-l  v 
2  21 
i=0 


3 .  Bayes  Method. 


2  Dj  +  1 

i=l  } 


—  +  0?.  +  ■  .  .  .  Dj  +  1 


2  Pj  + 


Pi  + 


T 

2  Pi 
1=1 


+  TRF 


(Di>  + - = -  (D2)  + 

l  P  I  t_i 

i=1  1  TRF 


<Dt)  + 


1=1  ?1  +  ™ 


2  Pi  + 
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<°1> + 


2  Pi  +  --- 

.  .  1  TRF 


2  P4  +  ~ 
i=1  1  TRF 


(D2)  + 


1 _ .  TRF 

T  (Ut;  +  TRF“ 

2  Pt  +  ~- 
i=1  1  TRF 


2  Pi  +  =£= 
.  -  1  TRF 


Substituting  Xi  for  Dj/Pi  (the  usage  rate)  and  simplifying  yields: 


T 

2  Pi 

i=l 


(Xt)  +  —  = - (X2)  + 

T  1 
£  P*  +  TRF 


- - (Xt)  + 


2  Pi  +  ~- 

1=1  TRF 


(TRF) 


where 


BRFt  -  the  forecast  for  the  T,h  period,  T  =  1,  2,  3,  . 
Dt  -  the  demand  for  the  Tth  period 
PT  =  the  average  population  for  the  Tth  period 
TRF  =  the  technical  replacement  factor 


NOTE :  The  Bayes  Method  is  similar  to  the  Ratio  Method.  There  are  two 

additional  terms.  One  is  in  the  denominator  of  the  usage  rate 


weights  — . 
6  TRF 


The  other  is  the  last  term 


(TRF) 


TRF  2  Pj  t  1 
i-1 


C-04 


Consequently,  changes  in  population  size  will  effect  the  Bayes 
Method  the  same  as  the  Ratio  Method.  The  difference  between  the 
two  methods  is  related  to  the  TRF.  Small  values  for  the  TRF  will 
add  a  large  constant  to  the  denominator  of  the  usage  rate  weights 
and  decrease  the  TRF  denominator  weight.  Therefore,  more  weight 
will  be  placed  on  the  TRF  than  on  the  usage  rates.  Large  values 
for  the  TRF  will  have  an  opposite  effect. 
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APPENDIX  D:  STATISTICAL  TESTS 


The  following  paragraphs  describe  the  statistical  tests  used  in  this 
study: 

1.  MEAN.  The  mean  is  used  to  measure  the  central  tendency  of  a  popula¬ 
tion  or  a  data  set. 


* 


X  = 


N 

E  XA 

i=l _ 

N 


Y 


M 

E  Ya 
i=l 
M 


where 

X  =  the  average  for  population/sample/variable  one 

-  the  ith  observation  for  the  first  population/sample/variable 
N  -  the  number  of  observations  for  the  first  population/sample/variable 
Y  -  the  average  lor  population/sample/variable  two 
Yx  =  the  ith  observation  for  the  second  population/sample/variable 
M  -  the  number  of  observations  for  the  second  popular ion/sample/var i ah  1 

2.  VARIANCE .  The  variance  and  standard  deviation  measure  how  a  papula 
tion  or  data  set  is  dispersed  around  its  mean. 

1  N 

S ?  -  fry  2  (X,  -  X)2 

i-1 

1  M 

Si  -  rpr  2  <Yt  -  Y)2 
1  i-1 
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where 


Sf  -  the  variance  for  the  first  population/sample/variable 

S|  -  the  variance  for  the  second  population/sample/variable 

3-  STANDARD  DEVIATION.  The  standard  deviation  is  the  square  root  of  the 
Variance . 

4.  F-TEST :  The  F  test  is  used  to  determine  whether  or  not  the  variance 
of  two  populations  are  the  same.  F  is  the  ratio  of  two  unknown  variances  from 
two  independent  normal  population/samples . 

S? 

si 

If  P<Fb_1<m_1)  >  a/2 ,  then  >  S§. 

If  then  Sf  <  S|. 

If  l-a/2  <  P(F„_j  <  a/2,  then  =  S|. 
where 

p(fn-i,m-i)  =  the  probability  of  observing  an  F  statistic  with  N-l  de¬ 
grees  of  freedom  in  the  numerator  and  M-l  degrees  of 
freedom  in  the  denominator 

a  -  the  level  of  significance 

5.  t-TEST  FOR  UNEQUAL  MEANS: 

This  test  is  used  here  to  determine  if  two  populations  have  different 
means . 
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t 


X  -  Y 


Si  si  ) 

~N  +  ~M 


Degrees  of  freedom  (d.f.)  =  (Sf/N  +  s|/M]/[ (Sf/N)2/(N- 


If  t  >  ta/2  d.f .  ,  then  X  >  Y. 

If  t  <  -ta/2  d.f.,  then  X  <  Y. 

If  -ta/2  d.f.  <  t  <  ta/2  d.f.,  then  X  =  Y. 

6.  PEARSON'S  CORRELATION  COEFFICIENT  fr'l  .  Pearson's 
efficient  (r)  is  used  to  measure  the  linear  relationship 
ables . 

r  =  1  indicates  a  positive /upward  linear  trend 
r  *  -1  indicates  a  negative/downward  linear  trend 
r  -  0  indicates  no  linear  trend 

where 


N 

2  (X,  -  X)  (Y4  -Y) 

i=l 


,,  N 

r N 

2  (X^X)2 

2  (Y,-Y)2 

i-l 

—i 

II 

•H 

i  )) 

M  and  N  are  assumed  equal 


If  r  >  Cc./2.N-2.  chen  r  “  I- 

!f  r  <  -ta/2,N-2.  thrn  r  =  -1. 

If  'ta/2,N-2  £  r  £  ta/2,N-2-  Chen  r  "  0 
where 

(W-2)*r 

cn-2  ”  r 

( I - r2) 


l)  +  (s|/m>2/(m-1) j 


correlation  co- 
between  two  vari - 


D-  03 


APPENDIX  E:  EQUIPMENT  ANALYSIS 


We  analyzed  the  impact  of  alternative  methods  on  an  engine  and  the  PHALANX 
weapon  system.  Since  the  performance  measurements  are  only  useful  for  com¬ 
parisons,  we  used  the  four  best  alternatives  for  all  items: 

•  Bayes  using  alternative  data  sources  when  3M  is  unavailable. 

•  Bayes  using  3M  data  only. 

•  Exponential  Smoothing  using  alternative  data  when  3M  is  unavailable  and 
an  average  data  starting  point,  and  smoothing  weights  of 

••  a  =  . 2 
•  •  a  —  .  4 

The  methods  available  under  Resystemization  (the  Current  Method  and  the 
Ratio  Method)  were  also  evaluated. 

The  performance  measurements  for  the  engine  are  displayed  in  TABLE  1  and 
for  the  PHALANX  in  TABLE  2.  The  results  are  very  close  for  the  four  alterna¬ 
tives  for  both  equipments.  All  four  alternatives  tend  to  generate  slightly 
better  performance  than  the  methods  available  under  Resystemization  (the  Cur¬ 
rent  Method  and  the  Ratio  Method).  For  the  engine,  the  results  are  consistent 
with  those  in  the  main  report.  The  proposed  method  generally  provides  the 
best  results  in  terms  of  stability  and  accuracy,  but  not  as  well  in  handling 
extremes.  For  the  PHALANX  data,  the  results  are  inconclusive. 
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TABLE  1 

SELECTED  EQUIPMENT 

ENGINE  -  1.589  ITEMS 
APL  #  L665360264 


BAYES 

ALT  DATA 

BAYES 

3M  ONLY 

EXPONENTIAL 
a  -  .  2 

EXPONENTIAL 
a  =  .4 

RATIO 

CURRENT 

SYSTEM 

#  ITEMS  AL¬ 
WAYS  W/I 
LIMITS 

130 

123 

117 

119 

107 

106 

MEAN 

3.6 

3.6 

2.7 

2.7 

3.6 

3.7 

STABILITY 

MEDIAN 

4 

4 

3 

3 

4 

4 

MODE 

4 

4 

3 

3 

4 

/t 

%  MODE 

81 

81 

88 

88 

81 

81 

STD 

1.3 

1.3 

0.8 

0.8 

1.2 

1.0 

MEAN 

.0137 

.0006 

-  .0079 

.0085 

.0148 

-  .011 

MEDIAN 

-  .0039 

-  .0096 

-  .0046 

-  .0046 

.011 

-  .0088 

ME 

MAX 

10.8 

10.9 

12.1 

11.2 

2.3 

6.7 

MIN 

-1.5 

-1.5 

-3.0 

-2.0 

-9.2 

-1.6 

STD 

0.4 

0.4 

0.4 

0.4 

0.3 

0.2 

MEAN 

1.0 

1.0 

1.0 

1.2 

1.3 

MEDIAN 

.00002 

.0001 

.00003 

.0002 

.00011 

MSE 

UP  QUART I LE 

.0002 

.0008 

.0003 

.0003 

.001 

.0008 

LR  QUART I LE 

.000005 

.000024 

.0000045 

.0000045 

.000025 

.000017 

STD 

30.9 

30.9 

30.3 

30.2 

32.9 

41 . 1 
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TABLE  2 


SELECTED  EQUIPMENT 


PHALANX  -  12.727  ITEMS 
APL  #  006090052 


BAYES 

ALT  DATA 

BAYES 

3M  ONLY 

EXPONENTIAL 
a  **  .2 

EXPONENTIAL 
a  -  .4 

RATIO 

CURRENT 

SYSTEM 

#  ITEMS  AL¬ 
WAYS  W/I 
LIMITS 

1,380 

1,354 

1,339 

1,327 

1,010 

8 

MEAN 

1.9 

1.9 

1.7 

1.7 

2.0 

2.9 

STABILITY 

MEDIAN 

0 

1 

1 

1 

1 

2 

MODE 

0 

0 

0 

0 

0 

i 

%  MODE 

51 

50 

49 

49 

37 

40 

STD 

2.4 

2.4 

2.0 

1.9 

2.2 

2.2 

MEAN 

0.8 

0.9 

1.1 

1.1 

-0.8 

0.6 

MEDIAN 

.0003 

.0005 

.0002 

.0002 

.0006 

.00004 

ME 

MAX 

2,177.4 

2,374.9 

3,206.0 

3,233.0 

15.5 

1,799.8 

MIN 

-21.5 

-26.9 

-143.7 

-104.3 

2,074.5 

-138.4 

STD 

41.7 

45.4 

61.5 

62.0 

39.7 

34 . 6 

MEAN 

30,201.6 

29,479.1 

28,209.1 

28,299.5 

30,584.6 

36,052.4 

MEDIAN 

.000089 

.000085 

.00011 

.00011 

.  00012 

.0001  1 

MSE 

UP  QUARTILE 

.0019 

.0018 

.0022 

.0022 

.0023 

.0021 

LR  QUARTILE 

.0000025 

.0000025 

.0000031 

.000003 

.000000014 

.0000029 

STD 

1,576,497 

1,538,795 

1,471,124 

1,476,455 

1,596,397 

1,881,857 

DOCUMENT  CONTROL  DATA  •  R  L  D 


mm 


<•  «  urin  <  .i/fin  (*/  fN/i-,  f  ••  •>/«  ..f  .  ■  (»,i/  |  .mi  /  m./.' 


i  f  inC  a  c  I  ivi  r»  (  c  ufpu^Jie  «it«r/iur  ^ 

Navy  Fleet  Material  Support  Office 
Operations  Analysis  Department  (Code  93) 
Mechanicsburg,  PA  17055-0787 


.1  hr  . -nr.rrd  »•  .Trrjff  r,  /..«</  (  ,  /.,  *  r>i  (iCil » 


ncpofl  r  stcu»i  rv  Cu*ssiFic  *  tion 

Unclassif ied 


,  (ctPom  ni(.£ 


BRF  Analysis  and  Improvement 


*-  DESCRIPTIVE  NOTES  (Type  at  report  end  inclusive  dates) 


V  auTmORiSI  (First  name,  middle  initiml,  last  name) 

Robert  E.  Maitland 


«.  REPORT  DATE 


26  December  1989 


•«.  CONTRACT  or  GRANT  no. 


7a.  TOTAL  no.  OP  PACES  76.  NO.  OP  REPS 


*a.  ORIGINATOR'S  REPORT  NUMBER!*) 


6.  project  no.  932 1-E65— 8207 


96.  OTHER  REPORT  NO(S!  (Any  other  numbers  that  may  be  as  si gned 
this  report) 


10.  DISTRIBUTION  STATEMENT 


Distribution  of  this  document  is  unlimited 


12-  SPONSORING  MILITARY  ACTIVITY 


13  A  BSTR  AC  1 


The  Best  Replacement  Factor  (BRF)  is  used  primarily  in  Coordinated  Shipboard  Allowance 
List  (COSAL)  and  load  list  requirements  determination.  Recently,  many  inventory  problems 
including  stock  shortages,  stock  excesses,  and  churn  have  been  blamed  on  the  BRF.  To 
identify  what  problems  are  associated  with  the  current  method  used  to  compute  the  BRF,  we 
took  a  multi*-faceted  approach.  First,  we  discussed"  the  two  forecasting  methods  available 
under  Resystemizat ion  and  then  analyzed  the  volatility  and  trend  of  the  current  BRF. 
Casualty  Report  (CASREP)  items  and  problem  equipments  were  included  in  the  analysis.  We 
then  examined  alternative  methods  to  forecast  the  I’.KK  and  compared  them  with  the  current 
methods.  The  best  alternative  was  selected  based  on  its  performance  in  terms  of  stability, 
accuracy  with  respect  to  bias  and  extremes,  and  the  magnitude  of  errors.  Performance  was 
evaluated  separately  for  items  identified  as  readiness  contributors.  Test  COSALs  were 
built  with  the  current  BRFs  and  the  proposed  BRFs  to  evaluate  the  impact  of  the  change. 

We  recommend  using  the  exponential  probability  distribution  to  adjust  the  Technical 
Replacement  Factor  (TRF)  during  the  development  period,  and  after  the  development  period, 
exponential  smoothing  with  a  data  windsorizing  system  to  handle  extreme  values. 
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U.  S.  Atlantic  Fleet 
Attn:  Code  42 
Norfolk,  VA  23511 

Commander 
Naval  Air  Force 
U.  S.  Pacific  Fleet 
Attn:  Code  44 

Naval  Air  Station,  North  Island 
San  Diego,  CA  92135 

Commander 

Naval  Logistics  Command 

U.  S.  Pacific  Fleet  (Code  4121) 

Pearl  Harbor,  HI  96860 

Center  for  Naval  Analyses 
2000  N.  Beauregard  Street 
Alexandria,  VA  22311 

Naval  Postgraduate  School 
Attn:  Alan  McMasters  (Code  54MG) 
Monterey,  CA  93940  (3  cys) 

Naval  Postgraduate  School 
Attn:  Thomas  Moore  (Code  54MR) 
Monterey,  CA  93940 

Superintendent  (Code  1424) 

Naval  Postgraduate  School 
Monterey,  CA  93940 

Naval  Postgraduate  School 
Attn:  Code  55  (MI) 

Monterey,  CA  93940 


Afloat  Studies 


Commander 

Naval  Surface  Forces 
U.  S.  Atlantic  Fleet 
Attn:  N72 
Norfolk,  VA  23511 

Commanding  Officer 
Naval  Supply  Corps  School 
Attn:  Code  43 
Athens,  GA  30605 

U.  S.  Army  Logistics  Management  Center 
Defense  Logistics  Studies  Information  Exchange 
Fort  Lee,  VA  23801 

Defense  Documentation  Center 

Cameron  Station 

Alexandria,  VA  22314  (2  cys) 

Defense  Technical  Information  Center 
Cameron  Station 
Alexandria,  VA  22314 


